Release 2021.06.01

[utils] Escape URLs in sanitized_Request, not sanitize_url
d2558234cf added escaping of URLs while sanitizing. However, `sanitize_url` may not always receive an actual URL. Eg: When using `yt-dlp "search query" --default-search ytsearch`, `search query` gets escaped to `search%20query` before being prefixed with `ytsearch:` which is not the intended behavior. So the escaping is moved to `sanitized_Request` instead.
2026-08-08 21:28:38 +03:00 · 2021-06-01 20:29:03 +05:30 · 2021-06-01 20:29:02 +05:30 · 2021-06-01 20:14:03 +05:30 · 2021-06-01 03:32:09 +05:30 · 2021-06-01 02:48:20 +05:30
113 changed files with 3187 additions and 1497 deletions
@@ -21,7 +21,7 @@ assignees: ''

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.04.22. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.05.20. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in https://github.com/yt-dlp/yt-dlp.
 - Search the bugtracker for similar issues: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
@@ -29,7 +29,7 @@ Carefully read and work through this check list in order to prevent the most com
 -->

 - [ ] I'm reporting a broken site support
- [ ] I've verified that I'm running yt-dlp version **2021.04.22**
+- [ ] I've verified that I'm running yt-dlp version **2021.05.20**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
 - [ ] I've searched the bugtracker for similar issues including closed ones
@@ -44,7 +44,7 @@ Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your com
 [debug] User config: []
 [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
 [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
- [debug] yt-dlp version 2021.04.22
+ [debug] yt-dlp version 2021.05.20
 [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
 [debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
 [debug] Proxy map: {}
@@ -21,7 +21,7 @@ assignees: ''

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.04.22. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.05.20. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that site you are requesting is not dedicated to copyright infringement, see https://github.com/yt-dlp/yt-dlp. yt-dlp does not support such sites. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
 - Search the bugtracker for similar site support requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
@@ -29,7 +29,7 @@ Carefully read and work through this check list in order to prevent the most com
 -->

 - [ ] I'm reporting a new site support request
- [ ] I've verified that I'm running yt-dlp version **2021.04.22**
+- [ ] I've verified that I'm running yt-dlp version **2021.05.20**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that none of provided URLs violate any copyrights
 - [ ] I've searched the bugtracker for similar site support requests including closed ones
@@ -21,13 +21,13 @@ assignees: ''

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.04.22. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.05.20. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Search the bugtracker for similar site feature requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
 - Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
 -->

 - [ ] I'm reporting a site feature request
- [ ] I've verified that I'm running yt-dlp version **2021.04.22**
+- [ ] I've verified that I'm running yt-dlp version **2021.05.20**
 - [ ] I've searched the bugtracker for similar site feature requests including closed ones


@@ -21,7 +21,7 @@ assignees: ''

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.04.22. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.05.20. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in https://github.com/yt-dlp/yt-dlp.
 - Search the bugtracker for similar issues: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
@@ -30,7 +30,7 @@ Carefully read and work through this check list in order to prevent the most com
 -->

 - [ ] I'm reporting a broken site support issue
- [ ] I've verified that I'm running yt-dlp version **2021.04.22**
+- [ ] I've verified that I'm running yt-dlp version **2021.05.20**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
 - [ ] I've searched the bugtracker for similar bug reports including closed ones
@@ -46,7 +46,7 @@ Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your com
 [debug] User config: []
 [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
 [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
- [debug] yt-dlp version 2021.04.22
+ [debug] yt-dlp version 2021.05.20
 [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
 [debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
 [debug] Proxy map: {}
@@ -21,13 +21,13 @@ assignees: ''

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.04.22. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.05.20. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Search the bugtracker for similar feature requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
 - Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
 -->

 - [ ] I'm reporting a feature request
- [ ] I've verified that I'm running yt-dlp version **2021.04.22**
+- [ ] I've verified that I'm running yt-dlp version **2021.05.20**
 - [ ] I've searched the bugtracker for similar feature requests including closed ones


@@ -132,7 +132,7 @@ jobs:
    - name: Upgrade pip and enable wheel support
      run: python -m pip install pip==19.1.1 setuptools==43.0.0 wheel==0.33.6
    - name: Install Requirements for 32 Bit
-      run: pip install pyinstaller==3.5 mutagen==1.42.0 pycryptodome==3.9.4
+      run: pip install pyinstaller==3.5 mutagen==1.42.0 pycryptodome==3.9.4 pefile==2019.4.18
    - name: Bump version
      id: bump_version
      run: python devscripts/update-version.py
@@ -1,3 +1,46 @@
+# Config
+*.conf
+*.spec
+cookies
+cookies.txt
+
+# Downloaded
+*.srt
+*.ttml
+*.sbv
+*.vtt
+*.flv
+*.mp4
+*.m4a
+*.m4v
+*.mp3
+*.3gp
+*.webm
+*.wav
+*.ape
+*.mkv
+*.swf
+*.part
+*.part-*
+*.ytdl
+*.dump
+*.frag
+*.frag.urls
+*.aria2
+*.swp
+*.ogg
+*.opus
+*.info.json
+*.live_chat.json
+*.jpg
+*.png
+*.webp
+*.annotations.xml
+*.description
+
+# Allow config/media files in testdata
+!test/testdata/**
+
 # Python
 *.pyc
 *.pyo
@@ -43,48 +86,6 @@ README.txt
 yt-dlp.zip
 *.exe

-# Downloaded
-*.srt
-*.ttml
-*.sbv
-*.vtt
-*.flv
-*.mp4
-*.m4a
-*.m4v
-*.mp3
-*.3gp
-*.webm
-*.wav
-*.ape
-*.mkv
-*.swf
-*.part
-*.part-*
-*.ytdl
-*.dump
-*.frag
-*.frag.urls
-*.aria2
-*.swp
-*.ogg
-*.opus
-*.info.json
-*.live_chat.json
-*.jpg
-*.png
-*.webp
-*.annotations.xml
-*.description
-
-# Config
-*.conf
-*.spec
-cookies
-cookies.txt
-
-
-
 # Text Editor / IDE
 .idea
 *.iml
@@ -44,3 +44,7 @@ Lamieur
 tsukumijima
 Hadi0609
 b5eff52
+craftingmod
+tpikonen
+tripulse
+king-millez
@@ -10,7 +10,7 @@
 * Commit to master as `Release <version>`
 * Push to origin/release using `git push origin master:release`
    build task will now run
-* Update version.py using devscripts\update-version.py
+* Update version.py using `devscripts\update-version.py`
 * Run `make issuetemplates`
 * Commit to master as `[version] update :ci skip all`
 * Push to origin/master
@@ -19,16 +19,96 @@
 -->


-### 2021.05.11
+### 2021.06.01

+* Merge youtube-dl: Upto [commit/d495292](https://github.com/ytdl-org/youtube-dl/commit/d495292852b6c2f1bd58bc2141ff2b0265c952cf)
+* Pre-check archive and filters during playlist extraction
+* Handle Basic Auth `user:pass` in URLs by [hhirtz](https://github.com/hhirtz) and [pukkandan](https://github.com/pukkandan)
+* [archiveorg] Add YoutubeWebArchiveIE by [colethedj](https://github.com/colethedj) and [alex-gedeon](https://github.com/alex-gedeon)
+* [fancode] Add extractor by [rmsmachine](https://github.com/rmsmachine)
+* [patreon] Support vimeo embeds by [rhsmachine](https://github.com/rhsmachine)
+* [Saitosan] Add new extractor by [llacb47](https://github.com/llacb47)
+* [ShemarooMe] Add extractor by [Ashish0804](https://github.com/Ashish0804) and [pukkandan](https://github.com/pukkandan)
+* [telemundo] Add extractor by [king-millez](https://github.com/king-millez)
+* [SonyLIV] Add SonyLIVSeriesIE and subtitle support by [Ashish0804](https://github.com/Ashish0804)
+* [Hotstar] Add HotStarSeriesIE by [Ashish0804](https://github.com/Ashish0804)
+* [Voot] Add VootSeriesIE by [Ashish0804](https://github.com/Ashish0804)
+* [vidio] Support login and premium videos by [MinePlayersPE](https://github.com/MinePlayersPE)
+* [fragment] When using `-N`, do not keep the fragment content in memory
+* [ffmpeg] Download and merge in a single step if possible
+* [ThumbnailsConvertor] Support conversion to `png` and make it the default by [louie-github](https://github.com/louie-github)
+* [VideoConvertor] Generalize with remuxer and allow conditional recoding
+* [EmbedThumbnail] Embed in `mp4`/`m4a` using mutagen by [tripulse](https://github.com/tripulse) and [pukkandan](https://github.com/pukkandan)
+* [EmbedThumbnail] Embed if any thumbnail was downloaded, not just the best
+* [EmbedThumbnail] Correctly escape filename
+* [update] replace self without launching a subprocess in windows
+* [update] Block further update for unsupported systems
+* Refactor `__process_playlist` by creating `LazyList`
+* Write messages to `stderr` when both `quiet` and `verbose`
+* Sanitize and sort playlist thumbnails
+* Remove `None` values from `info.json`
+* [extractor] Always prefer native hls downloader by default
+* [extractor] Skip subtitles without URI in m3u8 manifests by [hheimbuerger](https://github.com/hheimbuerger)
+* [extractor] Functions to parse `socket.io` response as `json` by [pukkandan](https://github.com/pukkandan) and [llacb47](https://github.com/llacb47)
+* [extractor] Allow `note=False` when extracting manifests
+* [utils] Escape URLs in `sanitized_Request`, not `sanitize_url`
+* [hls] Disable external downloader for `webtt`
+* [youtube] `/live` URLs should raise error if channel is not live
+* [youtube] Bug fixes
+* [zee5] Fix m3u8 formats' extension
+* [ard] Allow URLs without `-` before id by [olifre](https://github.com/olifre)
+* [cleanup] `YoutubeDL._match_entry`
+* [cleanup] Refactor updater
+* [cleanup] Refactor ffmpeg convertors
+* [cleanup] setup.py
+
+
+### 2021.05.20
+
+* **Youtube improvements**: 
+    * Support youtube music `MP`, `VL` and `browse` pages
+    * Extract more formats for youtube music by [craftingmod](https://github.com/craftingmod), [colethedj](https://github.com/colethedj) and [pukkandan](https://github.com/pukkandan)
+    * Extract multiple subtitles in same language by [pukkandan](https://github.com/pukkandan) and [tpikonen](https://github.com/tpikonen)
+    * Redirect channels that doesn't have a `videos` tab to their `UU` playlists
+    * Support in-channel search
+    * Sort audio-only formats correctly
+    * Always extract `maxresdefault` thumbnail
+    * Extract audio language
+    * Add subtitle language names by [nixxo](https://github.com/nixxo) and [tpikonen](https://github.com/tpikonen)
+    * Show alerts only from the final webpage
+    * Add `html5=1` param to `get_video_info` page requests by [colethedj](https://github.com/colethedj)
+    * Better message when login required
+* **Add option `--print`**: to print any field/template
+    * Deprecates: `--get-description`, `--get-duration`, `--get-filename`, `--get-format`, `--get-id`, `--get-thumbnail`, `--get-title`, `--get-url`
+* Field `additional_urls` to download additional videos from metadata using [`--parse-metadata`](https://github.com/yt-dlp/yt-dlp#modifying-metadata)
+* Merge youtube-dl: Upto [commit/dfbbe29](https://github.com/ytdl-org/youtube-dl/commit/dfbbe2902fc67f0f93ee47a8077c148055c67a9b)
+* Write thumbnail of playlist and add `pl_thumbnail` outtmpl key
+* [embedthumbnail] Add `flac` support and refactor `mutagen` code by [pukkandan](https://github.com/pukkandan) and [tripulse](https://github.com/tripulse)
+* [audius:artist] Add extractor by [king-millez](https://github.com/king-millez)
+* [parlview] Add extractor by [king-millez](https://github.com/king-millez)
+* [tenplay] Fix extractor by [king-millez](https://github.com/king-millez)
+* [rmcdecouverte] Generalize `_VALID_URL`
+* Add compat-option `no-attach-infojson`
+* Add field `name` for subtitles
+* Ensure `post_extract` and `pre_process` only run once
+* Fix `--check-formats` when there is network error
+* Standardize `write_debug` and `get_param`
+* [options] Alias `--write-comments`, `--no-write-comments`
+* [options] Refactor callbacks
+* [test:download] Only extract enough videos for `playlist_mincount`
+* [extractor] bugfix for when `compat_opts` is not given
+* [build] Fix x86 build by [shirt](https://github.com/shirt-dev)
+* [cleanup] code formatting, youtube tests and readme
+
+### 2021.05.11
 * **Deprecate support for python versions < 3.6**
-* **Subtitle extraction from manifests** by [fstirlitz](https://github.com/fstirlitz). See [be6202f12b97858b9d716e608394b51065d0419f](https://github.com/yt-dlp/yt-dlp/commit/be6202f12b97858b9d716e608394b51065d0419f) for details
+* **Subtitle extraction from manifests** by [fstirlitz](https://github.com/fstirlitz). See [be6202f](https://github.com/yt-dlp/yt-dlp/commit/be6202f12b97858b9d716e608394b51065d0419f) for details
 * **Improve output template:**
    * Allow slicing lists/strings using `field.start:end:step`
    * A field can also be used as offset like `field1+num+field2`
    * A default value can be given using `field|default`
    * Prevent invalid fields from causing errors
-* Merge youtube-dl: Upto [commit/a726009](https://github.com/ytdl-org/youtube-dl/commit/a7260099873acc6dc7d76cafad2f6b139087afd0)
+* **Merge youtube-dl**: Upto [commit/a726009](https://github.com/ytdl-org/youtube-dl/commit/a7260099873acc6dc7d76cafad2f6b139087afd0)
 * **Remove options** `-l`, `-t`, `-A` completely and disable `--auto-number`, `--title`, `--literal`, `--id`
 * [Plugins] Prioritize plugins over standard extractors and prevent plugins from overwriting the standard extractor classes
 * [downloader] Fix `quiet` and `to_stderr`
@@ -63,7 +143,7 @@
 * [zee5] Fix py2 compatibility
 * Fix `playlist_index` and add `playlist_autonumber`. See [#302](https://github.com/yt-dlp/yt-dlp/issues/302) for details
 * Add experimental option `--check-formats` to test the URLs before format selection
-* Option `--compat-options` to revert some of yt-dlp's changes
+* Option `--compat-options` to revert [some of yt-dlp's changes](https://github.com/yt-dlp/yt-dlp#differences-in-default-behavior)
    * Deprecates `--list-formats-as-table`, `--list-formats-old`
 * Fix number of digits in `%(playlist_index)s`
 * Fix case sensitivity of format selector
@@ -574,4 +654,4 @@
 * [generic] Extract embedded youtube and twitter videos by [diegorodriguezv](https://github.com/diegorodriguezv)
 * [ffmpeg] Ensure all streams are copied by [pukkandan](https://github.com/pukkandan)
 * [embedthumbnail] Fix for os.rename error by [pukkandan](https://github.com/pukkandan)
-* make_win.bat: don't use UPX to pack vcruntime140.dll by [jbruchon](https://github.com/jbruchon)
+* make_win.bat: don't use UPX to pack vcruntime140.dll by [jbruchon](https://github.com/jbruchon)
@@ -66,15 +66,17 @@ The major new features from the latest release of [blackjack4494/yt-dlc](https:/

 * **[Format Sorting](#sorting-formats)**: The default format sorting options have been changed so that higher resolution and better codecs will be now preferred instead of simply using larger bitrate. Furthermore, you can now specify the sort order using `-S`. This allows for much easier format selection that what is possible by simply using `--format` ([examples](#format-selection-examples))

-* **Merged with youtube-dl [commit/a726009](https://github.com/ytdl-org/youtube-dl/commit/a7260099873acc6dc7d76cafad2f6b139087afd0)**: (v2021.04.26) You get all the latest features and patches of [youtube-dl](https://github.com/ytdl-org/youtube-dl) in addition to all the features of [youtube-dlc](https://github.com/blackjack4494/yt-dlc)
+* **Merged with youtube-dl [commit/d495292](https://github.com/ytdl-org/youtube-dl/commit/d495292852b6c2f1bd58bc2141ff2b0265c952cf)**: (v2021.05.16) You get all the latest features and patches of [youtube-dl](https://github.com/ytdl-org/youtube-dl) in addition to all the features of [youtube-dlc](https://github.com/blackjack4494/yt-dlc)

-* **Merged with animelover1984/youtube-dl**: You get most of the features and improvements from [animelover1984/youtube-dl](https://github.com/animelover1984/youtube-dl) including `--get-comments`, `BiliBiliSearch`, `BilibiliChannel`, Embedding thumbnail in mp4/ogg/opus, playlist infojson etc. Note that the NicoNico improvements are not available. See [#31](https://github.com/yt-dlp/yt-dlp/pull/31) for details.
+* **Merged with animelover1984/youtube-dl**: You get most of the features and improvements from [animelover1984/youtube-dl](https://github.com/animelover1984/youtube-dl) including `--write-comments`, `BiliBiliSearch`, `BilibiliChannel`, Embedding thumbnail in mp4/ogg/opus, playlist infojson etc. Note that the NicoNico improvements are not available. See [#31](https://github.com/yt-dlp/yt-dlp/pull/31) for details.

 * **Youtube improvements**:
-    * All Youtube Feeds (`:ytfav`, `:ytwatchlater`, `:ytsubs`, `:ythistory`, `:ytrec`) works and supports downloading multiple pages of content
-    * Youtube search (`ytsearch:`, `ytsearchdate:`) along with Search URLs work
-    * Youtube mixes supports downloading multiple pages of content
+    * All Feeds (`:ytfav`, `:ytwatchlater`, `:ytsubs`, `:ythistory`, `:ytrec`) supports downloading multiple pages of content
+    * Search (`ytsearch:`, `ytsearchdate:`), search URLs and in-channel search works
+    * Mixes supports downloading multiple pages of content
    * Redirect channel's home URL automatically to `/video` to preserve the old behaviour
+    * `255kbps` audio is extracted from youtube music if premium cookies are given
+    * Youtube music Albums, channels etc can be downloaded

 * **Split video by chapters**: Videos can be split into multiple files based on chapters using `--split-chapters`

@@ -82,11 +84,11 @@ The major new features from the latest release of [blackjack4494/yt-dlc](https:/

 * **Aria2c with HLS/DASH**: You can use `aria2c` as the external downloader for DASH(mpd) and HLS(m3u8) formats

-* **New extractors**: AnimeLab, Philo MSO, Rcs, Gedi, bitwave.tv, mildom, audius, zee5, mtv.it, wimtv, pluto.tv, niconico users, discoveryplus.in, mediathek, NFHSNetwork, nebula, ukcolumn, whowatch, MxplayerShow
+* **New extractors**: AnimeLab, Philo MSO, Rcs, Gedi, bitwave.tv, mildom, audius, zee5, mtv.it, wimtv, pluto.tv, niconico users, discoveryplus.in, mediathek, NFHSNetwork, nebula, ukcolumn, whowatch, MxplayerShow, parlview (au), YoutubeWebArchive, fancode, Saitosan, ShemarooMe, telemundo, VootSeries, SonyLIVSeries, HotstarSeries

-* **Fixed extractors**: archive.org, roosterteeth.com, skyit, instagram, itv, SouthparkDe, spreaker, Vlive, akamai, ina, rumble, tennistv, amcnetworks, la7 podcasts, linuxacadamy, nitter, twitcasting, viu, crackle, curiositystream, mediasite, rmcdecouverte, sonyliv, tubi
+* **Fixed extractors**: archive.org, roosterteeth.com, skyit, instagram, itv, SouthparkDe, spreaker, Vlive, akamai, ina, rumble, tennistv, amcnetworks, la7 podcasts, linuxacadamy, nitter, twitcasting, viu, crackle, curiositystream, mediasite, rmcdecouverte, sonyliv, tubi, tenplay, patreon

-* **Subtitle extraction from manifests**: Subtitles can be extracted from streaming media manifests. See [be6202f12b97858b9d716e608394b51065d0419f](https://github.com/yt-dlp/yt-dlp/commit/be6202f12b97858b9d716e608394b51065d0419f) for details
+* **Subtitle extraction from manifests**: Subtitles can be extracted from streaming media manifests. See [commit/be6202f](https://github.com/yt-dlp/yt-dlp/commit/be6202f12b97858b9d716e608394b51065d0419f) for details

 * **Multiple paths and output templates**: You can give different [output templates](#output-template) and download paths for different types of files. You can also set a temporary path where intermediary files are downloaded to using `--paths` (`-P`)

@@ -114,24 +116,26 @@ If you are coming from [youtube-dl](https://github.com/ytdl-org/youtube-dl), the

 Some of yt-dlp's default options are different from that of youtube-dl and youtube-dlc.

-1. The options `--id`, `--auto-number` (`-A`), `--title` (`-t`) and `--literal` (`-l`), no longer work. See [removed options](#Removed) for details
-1. `avconv` is not supported as as an alternative to `ffmpeg`
-1. The default [output template](#output-template) is `%(title)s [%(id)s].%(ext)s`. There is no real reason for this change. This was changed before yt-dlp was ever made public and now there are no plans to change it back to `%(title)s.%(id)s.%(ext)s`. Instead, you may use `--compat-options filename`
-1. The default [format sorting](sorting-formats) is different from youtube-dl and prefers higher resolution and better codecs rather than higher bitrates. You can use the `--format-sort` option to change this to any order you prefer, or use `--compat-options format-sort` to use youtube-dl's sorting order
-1. The default format selector is `bv*+ba/b`. This means that if a combined video + audio format that is better than the best video-only format is found, the former will be prefered. Use `-f bv+ba/b` or `--compat-options format-spec` to revert this
-1. Unlike youtube-dlc, yt-dlp does not allow merging multiple audio/video streams into one file by default (since this conflicts with the use of `-f bv*+ba`). If needed, this feature must be enabled using `--audio-multistreams` and `--video-multistreams`. You can also use `--compat-options multistreams` to enable both
-1. `--ignore-errors` is enabled by default. Use `--abort-on-error` or `--compat-options abort-on-error` to abort on errors instead
-1. When writing metadata files such as thumbnails, description or infojson, the same information (if available) is also written for playlists. Use `--no-write-playlist-metafiles` or `--compat-options no-playlist-metafiles` to not write these files
-1. `playlist_index` behaves differently when used with options like `--playlist-reverse` and `--playlist-items`. See [#302](https://github.com/yt-dlp/yt-dlp/issues/302) for details. You can use `--compat-options playlist-index` if you want to keep the earlier behavior
-1. The output of `-F` is listed in a new format. Use `--compat-options list-formats` to revert this
-1. Youtube live chat (if available) is considered as a subtitle. Use `--sub-langs all,-live_chat` to download all subtitles except live chat. You can also use `--compat-options no-live-chat` to prevent live chat from downloading
-1. Youtube channel URLs are automatically redirected to `/video`. Either append a `/featured` to the URL or use `--compat-options no-youtube-channel-redirect` to download only the videos in the home page
-1. Unavailable videos are also listed for youtube playlists. Use `--compat-options no-youtube-unavailable-videos` to remove this
+* The options `--id`, `--auto-number` (`-A`), `--title` (`-t`) and `--literal` (`-l`), no longer work. See [removed options](#Removed) for details
+* `avconv` is not supported as as an alternative to `ffmpeg`
+* The default [output template](#output-template) is `%(title)s [%(id)s].%(ext)s`. There is no real reason for this change. This was changed before yt-dlp was ever made public and now there are no plans to change it back to `%(title)s.%(id)s.%(ext)s`. Instead, you may use `--compat-options filename`
+* The default [format sorting](sorting-formats) is different from youtube-dl and prefers higher resolution and better codecs rather than higher bitrates. You can use the `--format-sort` option to change this to any order you prefer, or use `--compat-options format-sort` to use youtube-dl's sorting order
+* The default format selector is `bv*+ba/b`. This means that if a combined video + audio format that is better than the best video-only format is found, the former will be prefered. Use `-f bv+ba/b` or `--compat-options format-spec` to revert this
+* Unlike youtube-dlc, yt-dlp does not allow merging multiple audio/video streams into one file by default (since this conflicts with the use of `-f bv*+ba`). If needed, this feature must be enabled using `--audio-multistreams` and `--video-multistreams`. You can also use `--compat-options multistreams` to enable both
+* `--ignore-errors` is enabled by default. Use `--abort-on-error` or `--compat-options abort-on-error` to abort on errors instead
+* When writing metadata files such as thumbnails, description or infojson, the same information (if available) is also written for playlists. Use `--no-write-playlist-metafiles` or `--compat-options no-playlist-metafiles` to not write these files
+* `--add-metadata` attaches the `infojson` to `mkv` files in addition to writing the metadata when used with `--write-infojson`. Use `--compat-options no-attach-info-json` to revert this
+* `playlist_index` behaves differently when used with options like `--playlist-reverse` and `--playlist-items`. See [#302](https://github.com/yt-dlp/yt-dlp/issues/302) for details. You can use `--compat-options playlist-index` if you want to keep the earlier behavior
+* The output of `-F` is listed in a new format. Use `--compat-options list-formats` to revert this
+* Youtube live chat (if available) is considered as a subtitle. Use `--sub-langs all,-live_chat` to download all subtitles except live chat. You can also use `--compat-options no-live-chat` to prevent live chat from downloading
+* Youtube channel URLs are automatically redirected to `/video`. Append a `/featured` to the URL to download only the videos in the home page. If the channel does not have a videos tab, we try to download the equivalent `UU` playlist instead. Also, `/live` URLs raise an error if there are no live videos instead of silently downloading the entire channel. You may use `--compat-options no-youtube-channel-redirect` to revert all these redirections
+* Unavailable videos are also listed for youtube playlists. Use `--compat-options no-youtube-unavailable-videos` to remove this
+* If `ffmpeg` is used as the downloader, the downloading and merging of formats happen in a single step when possible. Use `--compat-options no-direct-merge` to revert this

 For ease of use, a few more compat options are available:
-1. `--compat-options all` = Use all compat options
-1. `--compat-options youtube-dl` = `--compat-options all,-multistreams`
-1. `--compat-options youtube-dlc` = `--compat-options all,-no-live-chat,-no-youtube-channel-redirect`
+* `--compat-options all`: Use all compat options
+* `--compat-options youtube-dl`: Same as `--compat-options all,-multistreams`
+* `--compat-options youtube-dlc`: Same as `--compat-options all,-no-live-chat,-no-youtube-channel-redirect`


 # INSTALLATION
@@ -167,7 +171,7 @@ Python versions 3.6+ (CPython and PyPy) are officially supported. Other versions

 On windows, [Microsoft Visual C++ 2010 Redistributable Package (x86)](https://www.microsoft.com/en-us/download/details.aspx?id=26999) is also necessary to run yt-dlp. You probably already have this, but if the executable throws an error due to missing `MSVCR100.dll` you need to install it.

-Although there are no other required dependencies, `ffmpeg` and `ffprobe` are highly recommended. Other optional dependencies are `sponskrub`, `AtomicParsley`, `mutagen`, `pycryptodome` and any of the supported external downloaders. Note that the windows releases are already built with the python interpreter, mutagen and pycryptodome included.
+Although there are no other required dependencies, `ffmpeg` and `ffprobe` are highly recommended. Other optional dependencies are `sponskrub`, `AtomicParsley`, `mutagen`, `pycryptodome`, `phantomjs` and any of the supported external downloaders. Note that the windows releases are already built with the python interpreter, mutagen and pycryptodome included.

 ### UPDATE
 You can use `yt-dlp -U` to update if you are using the provided release.
@@ -482,10 +486,13 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
                                     could still contain some personal
                                     information (default)
    --no-clean-infojson              Write all fields to the infojson
-    --get-comments                   Retrieve video comments to be placed in the
-                                     .info.json file. The comments are fetched
-                                     even without this option if the extraction
-                                     is known to be quick
+    --write-comments                 Retrieve video comments to be placed in the
+                                     infojson. The comments are fetched even
+                                     without this option if the extraction is
+                                     known to be quick (Alias: --get-comments)
+    --no-write-comments              Do not retrieve video comments unless the
+                                     extraction is known to be quick
+                                     (Alias: --no-get-comments)
    --load-info-json FILE            JSON file containing the video information
                                     (created with the "--write-info-json"
                                     option)
@@ -533,14 +540,9 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
                                     formats are found (default)
    --skip-download                  Do not download the video but write all
                                     related files (Alias: --no-download)
-    -g, --get-url                    Simulate, quiet but print URL
-    -e, --get-title                  Simulate, quiet but print title
-    --get-id                         Simulate, quiet but print id
-    --get-thumbnail                  Simulate, quiet but print thumbnail URL
-    --get-description                Simulate, quiet but print video description
-    --get-duration                   Simulate, quiet but print video length
-    --get-filename                   Simulate, quiet but print output filename
-    --get-format                     Simulate, quiet but print output format
+    -O, --print TEMPLATE             Simulate, quiet but print the given fields.
+                                     Either a field name or similar formatting
+                                     as the output template can be used
    -j, --dump-json                  Simulate, quiet but print JSON information.
                                     See "OUTPUT TEMPLATE" for a description of
                                     available keys
@@ -669,10 +671,10 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
 ## Post-Processing Options:
    -x, --extract-audio              Convert video files to audio-only files
                                     (requires ffmpeg and ffprobe)
-    --audio-format FORMAT            Specify audio format: "best", "aac",
-                                     "flac", "mp3", "m4a", "opus", "vorbis", or
-                                     "wav"; "best" by default; No effect without
-                                     -x
+    --audio-format FORMAT            Specify audio format to convert the audio
+                                     to when -x is used. Currently supported
+                                     formats are: best (default) or one of
+                                     aac|flac|mp3|m4a|opus|vorbis|wav
    --audio-quality QUALITY          Specify ffmpeg audio quality, insert a
                                     value between 0 (better) and 9 (worse) for
                                     VBR or a specific bitrate like 128K
@@ -682,12 +684,12 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
                                     |webm|mov|avi|mp3|mka|m4a|ogg|opus). If
                                     target container does not support the
                                     video/audio codec, remuxing will fail. You
-                                     can specify multiple rules; eg.
+                                     can specify multiple rules; Eg.
                                     "aac>m4a/mov>mp4/mkv" will remux aac to
                                     m4a, mov to mp4 and anything else to mkv.
    --recode-video FORMAT            Re-encode the video into another format if
-                                     re-encoding is necessary. The supported
-                                     formats are the same as --remux-video
+                                     re-encoding is necessary. The syntax and
+                                     supported formats are the same as --remux-video
    --postprocessor-args NAME:ARGS   Give these arguments to the postprocessors.
                                     Specify the postprocessor/executable name
                                     and the arguments separated by a colon ":"
@@ -748,10 +750,10 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
                                     fields are passed, "%(filepath)s" is
                                     appended to the end of the command
    --convert-subs FORMAT            Convert the subtitles to another format
-                                     (currently supported: srt|ass|vtt|lrc)
+                                     (currently supported: srt|vtt|ass|lrc)
                                     (Alias: --convert-subtitles)
    --convert-thumbnails FORMAT      Convert the thumbnails to another format
-                                     (currently supported: jpg)
+                                     (currently supported: jpg|png)
    --split-chapters                 Split video into multiple files based on
                                     internal chapters. The "chapter:" prefix
                                     can be used with "--paths" and "--output"
@@ -822,7 +824,7 @@ You can configure yt-dlp by placing any supported command line option to a confi
    * `~/yt-dlp.conf.txt`

    Note that `~` points to `C:\Users\<user name>` on windows. Also, `%XDG_CONFIG_HOME%` defaults to `~/.config` if undefined
-1. **System Configuration**: `/etc/yt-dlp.conf` or `/etc/yt-dlp.conf`
+1. **System Configuration**: `/etc/yt-dlp.conf`

 For example, with the following configuration file yt-dlp will always extract the audio, not copy the mtime, use a proxy and save all videos under `YouTube` directory in your home directory:
 ```
@@ -889,7 +891,7 @@ To summarize, the general syntax for a field is:
 %(name[.keys][addition][>strf][|default])[flags][width][.precision][length]type
 ```

-Additionally, you can set different output templates for the various metadata files separately from the general output template by specifying the type of file followed by the template separated by a colon `:`. The different file types supported are `subtitle`, `thumbnail`, `description`, `annotation`, `infojson`, `pl_description`, `pl_infojson`, `chapter`. For example, `-o '%(title)s.%(ext)s' -o 'thumbnail:%(title)s\%(title)s.%(ext)s'`  will put the thumbnails in a folder with the same name as the video.
+Additionally, you can set different output templates for the various metadata files separately from the general output template by specifying the type of file followed by the template separated by a colon `:`. The different file types supported are `subtitle`, `thumbnail`, `description`, `annotation`, `infojson`, `pl_thumbnail`, `pl_description`, `pl_infojson`, `chapter`. For example, `-o '%(title)s.%(ext)s' -o 'thumbnail:%(title)s\%(title)s.%(ext)s'`  will put the thumbnails in a folder with the same name as the video.

 The available fields are:

@@ -911,7 +913,7 @@ The available fields are:
 - `channel_id` (string): Id of the channel
 - `location` (string): Physical location where the video was filmed
 - `duration` (numeric): Length of the video in seconds
- - `duration_string` (string): Length of the video (HH-mm-ss)
+ - `duration_string` (string): Length of the video (HH:mm:ss)
 - `view_count` (numeric): How many users have watched the video on the platform
 - `like_count` (numeric): Number of positive ratings of the video
 - `dislike_count` (numeric): Number of negative ratings of the video
@@ -989,6 +991,11 @@ Available for `chapter:` prefix when using `--split-chapters` for videos with in
 - `section_start` (numeric): Start time of the chapter in seconds
 - `section_end` (numeric): End time of the chapter in seconds

+Available only when used in `--print`:
+
+ - `urls` (string): The URLs of all requested formats, one in each line
+ - `filename` (string): Name of the video file. Note that the actual filename may be different due to post-processing. Use `--exec echo` to get the name after all postprocessing is complete
+
 Each aforementioned sequence when referenced in an output template will be replaced by the actual value corresponding to the sequence name. Note that some of the sequences are not guaranteed to be present since they depend on the metadata obtained by a particular extractor. Such sequences will be replaced with placeholder value provided with `--output-na-placeholder` (`NA` by default).

 For example for `-o %(title)s-%(id)s.%(ext)s` and an mp4 video with title `yt-dlp test video` and id `BaW_jenozKcj`, this will result in a `yt-dlp test video-BaW_jenozKcj.mp4` file created in the current directory.
@@ -1278,7 +1285,9 @@ The metadata obtained the the extractors can be modified by using `--parse-metad

 Note that any field created by this can be used in the [output template](#output-template) and will also affect the media file's metadata added when using `--add-metadata`.

-You can also use this to change only the metadata that is embedded in the media file. To do this, set the value of the corresponding field with a `meta_` prefix. For example, any value you set to `meta_description` field will be added to the `description` field in the file. You can use this to set a different "description" and "synopsis", for example.
+This option also has a few special uses:
+* You can use this to change the metadata that is embedded in the media file. To do this, set the value of the corresponding field with a `meta_` prefix. For example, any value you set to `meta_description` field will be added to the `description` field in the file. You can use this to set a different "description" and "synopsis", for example
+* You can download an additional URL based on the metadata of the currently downloaded video. To do this, set the field `additional_urls` to the URL that you want to download. Eg: `--parse-metadata "description:(?P<additional_urls>https?://www\.vimeo\.com/\d+)` will download the first vimeo video found in the description

 ## Modifying metadata examples

@@ -1312,6 +1321,14 @@ These are all the deprecated options and the current alternative to achieve the
 #### Not recommended
 While these options still work, their use is not recommended since there are other alternatives to achieve the same

+    --get-description                --print description
+    --get-duration                   --print duration_string
+    --get-filename                   --print filename
+    --get-format                     --print format
+    --get-id                         --print id
+    --get-thumbnail                  --print thumbnail
+    -e, --get-title                  --print title
+    -g, --get-url                    --print urls
    --all-formats                    -f all
    --all-subs                       --sub-langs all --write-subs
    --autonumber-size NUMBER         Use string formatting. Eg: %(autonumber)03d
@@ -1323,6 +1340,7 @@ While these options still work, their use is not recommended since there are oth
    --list-formats-as-table          --compat-options -list-formats [Default] (Alias: --no-list-formats-old)
    --sponskrub-args ARGS            --ppa "sponskrub:ARGS"
    --test                           Used by developers for testing extractors. Not intended for the end user
+    --youtube-print-sig-code         Used for testing youtube signatures


 #### Old aliases
@@ -1353,7 +1371,6 @@ These options may no longer work as intended
    --no-call-home                   Default
    --include-ads                    No longer supported
    --no-include-ads                 Default
-    --youtube-print-sig-code         No longer supported

 #### Removed
 These options were deprecated since 2014 and have now been entirely removed
@@ -17,7 +17,7 @@ assert arch in ('32', '64')
 print('Building %sbit version' % arch)
 _x86 = '_x86' if arch == '32' else ''

-FILE_DESCRIPTION = 'Media Downloader%s' % (' (32 Bit)' if _x86 else '')
+FILE_DESCRIPTION = 'yt-dlp%s' % (' (32 Bit)' if _x86 else '')

 # root_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
 # print('Changing working directory to %s' % root_dir)
@@ -9,45 +9,44 @@ from distutils.spawn import spawn


 # Get the version from yt_dlp/version.py without importing the package
-exec(compile(open('yt_dlp/version.py').read(),
-             'yt_dlp/version.py', 'exec'))
+exec(compile(open('yt_dlp/version.py').read(), 'yt_dlp/version.py', 'exec'))


 DESCRIPTION = 'Command-line program to download videos from YouTube.com and many other other video platforms.'

 LONG_DESCRIPTION = '\n\n'.join((
    'Official repository: <https://github.com/yt-dlp/yt-dlp>',
-    '**PS**: Many links in this document will not work since this is a copy of the README.md from Github',
+    '**PS**: Some links in this document will not work since this is a copy of the README.md from Github',
    open('README.md', 'r', encoding='utf-8').read()))

 REQUIREMENTS = ['mutagen', 'pycryptodome']

+if sys.argv[1:2] == ['py2exe']:
+    raise NotImplementedError('py2exe is not currently supported; instead, use "pyinst.py" to build with pyinstaller')

-if len(sys.argv) >= 2 and sys.argv[1] == 'py2exe':
-    print('inv')
-else:
-    files_spec = [
-        ('share/bash-completion/completions', ['completions/bash/yt-dlp']),
-        ('share/zsh/site-functions', ['completions/zsh/_yt-dlp']),
-        ('share/fish/vendor_completions.d', ['completions/fish/yt-dlp.fish']),
-        ('share/doc/yt_dlp', ['README.txt']),
-        ('share/man/man1', ['yt-dlp.1'])
-    ]
-    root = os.path.dirname(os.path.abspath(__file__))
-    data_files = []
-    for dirname, files in files_spec:
-        resfiles = []
-        for fn in files:
-            if not os.path.exists(fn):
-                warnings.warn('Skipping file %s since it is not present. Try running `make pypi-files` first.' % fn)
-            else:
-                resfiles.append(fn)
-        data_files.append((dirname, resfiles))

-    params = {
-        'data_files': data_files,
-    }
-    params['entry_points'] = {'console_scripts': ['yt-dlp = yt_dlp:main']}
+files_spec = [
+    ('share/bash-completion/completions', ['completions/bash/yt-dlp']),
+    ('share/zsh/site-functions', ['completions/zsh/_yt-dlp']),
+    ('share/fish/vendor_completions.d', ['completions/fish/yt-dlp.fish']),
+    ('share/doc/yt_dlp', ['README.txt']),
+    ('share/man/man1', ['yt-dlp.1'])
+]
+root = os.path.dirname(os.path.abspath(__file__))
+data_files = []
+for dirname, files in files_spec:
+    resfiles = []
+    for fn in files:
+        if not os.path.exists(fn):
+            warnings.warn('Skipping file %s since it is not present. Try running `make pypi-files` first' % fn)
+        else:
+            resfiles.append(fn)
+    data_files.append((dirname, resfiles))
+
+params = {
+    'data_files': data_files,
+}
+params['entry_points'] = {'console_scripts': ['yt-dlp = yt_dlp:main']}


 class build_lazy_extractors(Command):
@@ -61,10 +60,8 @@ class build_lazy_extractors(Command):
        pass

    def run(self):
-        spawn(
-            [sys.executable, 'devscripts/make_lazy_extractors.py', 'yt_dlp/extractor/lazy_extractors.py'],
-            dry_run=self.dry_run,
-        )
+        spawn([sys.executable, 'devscripts/make_lazy_extractors.py', 'yt_dlp/extractor/lazy_extractors.py'],
+              dry_run=self.dry_run)


 packages = find_packages(exclude=('youtube_dl', 'test', 'ytdlp_plugins'))
@@ -82,6 +82,7 @@
 - **audiomack**
 - **audiomack:album**
 - **Audius**: Audius.co
+ - **audius:artist**: Audius.co profile/artist pages
 - **audius:playlist**: Audius.co playlists
 - **audius:track**: Audius track ID or API link. Prepend with "audius:"
 - **AWAAN**
@@ -306,6 +307,7 @@
 - **EyedoTV**
 - **facebook**
 - **FacebookPluginsVideo**
+ - **fancode:vod**
 - **faz.net**
 - **fc2**
 - **fc2:embed**
@@ -391,6 +393,7 @@
 - **HotNewHipHop**
 - **hotstar**
 - **hotstar:playlist**
+ - **hotstar:series**
 - **Howcast**
 - **HowStuffWorks**
 - **hrfernsehen**
@@ -725,6 +728,7 @@
 - **pandora.tv**: 판도라TV
 - **ParamountNetwork**
 - **parliamentlive.tv**: UK parliament videos
+ - **Parlview**
 - **Patreon**
 - **pbs**: Public Broadcasting Service (PBS) and member stations: PBS: Public Broadcasting Service, APT - Alabama Public Television (WBIQ), GPB/Georgia Public Broadcasting (WGTV), Mississippi Public Broadcasting (WMPN), Nashville Public Television (WNPT), WFSU-TV (WFSU), WSRE (WSRE), WTCI (WTCI), WPBA/Channel 30 (WPBA), Alaska Public Media (KAKM), Arizona PBS (KAET), KNME-TV/Channel 5 (KNME), Vegas PBS (KLVX), AETN/ARKANSAS ETV NETWORK (KETS), KET (WKLE), WKNO/Channel 10 (WKNO), LPB/LOUISIANA PUBLIC BROADCASTING (WLPB), OETA (KETA), Ozarks Public Television (KOZK), WSIU Public Broadcasting (WSIU), KEET TV (KEET), KIXE/Channel 9 (KIXE), KPBS San Diego (KPBS), KQED (KQED), KVIE Public Television (KVIE), PBS SoCal/KOCE (KOCE), ValleyPBS (KVPT), CONNECTICUT PUBLIC TELEVISION (WEDH), KNPB Channel 5 (KNPB), SOPTV (KSYS), Rocky Mountain PBS (KRMA), KENW-TV3 (KENW), KUED Channel 7 (KUED), Wyoming PBS (KCWC), Colorado Public Television / KBDI 12 (KBDI), KBYU-TV (KBYU), Thirteen/WNET New York (WNET), WGBH/Channel 2 (WGBH), WGBY (WGBY), NJTV Public Media NJ (WNJT), WLIW21 (WLIW), mpt/Maryland Public Television (WMPB), WETA Television and Radio (WETA), WHYY (WHYY), PBS 39 (WLVT), WVPT - Your Source for PBS and More! (WVPT), Howard University Television (WHUT), WEDU PBS (WEDU), WGCU Public Media (WGCU), WPBT2 (WPBT), WUCF TV (WUCF), WUFT/Channel 5 (WUFT), WXEL/Channel 42 (WXEL), WLRN/Channel 17 (WLRN), WUSF Public Broadcasting (WUSF), ETV (WRLK), UNC-TV (WUNC), PBS Hawaii - Oceanic Cable Channel 10 (KHET), Idaho Public Television (KAID), KSPS (KSPS), OPB (KOPB), KWSU/Channel 10 & KTNW/Channel 31 (KWSU), WILL-TV (WILL), Network Knowledge - WSEC/Springfield (WSEC), WTTW11 (WTTW), Iowa Public Television/IPTV (KDIN), Nine Network (KETC), PBS39 Fort Wayne (WFWA), WFYI Indianapolis (WFYI), Milwaukee Public Television (WMVS), WNIN (WNIN), WNIT Public Television (WNIT), WPT (WPNE), WVUT/Channel 22 (WVUT), WEIU/Channel 51 (WEIU), WQPT-TV (WQPT), WYCC PBS Chicago (WYCC), WIPB-TV (WIPB), WTIU (WTIU), CET  (WCET), ThinkTVNetwork (WPTD), WBGU-TV (WBGU), WGVU TV (WGVU), NET1 (KUON), Pioneer Public Television (KWCM), SDPB Television (KUSD), TPT (KTCA), KSMQ (KSMQ), KPTS/Channel 8 (KPTS), KTWU/Channel 11 (KTWU), East Tennessee PBS (WSJK), WCTE-TV (WCTE), WLJT, Channel 11 (WLJT), WOSU TV (WOSU), WOUB/WOUC (WOUB), WVPB (WVPB), WKYU-PBS (WKYU), KERA 13 (KERA), MPBN (WCBB), Mountain Lake PBS (WCFE), NHPTV (WENH), Vermont PBS (WETK), witf (WITF), WQED Multimedia (WQED), WMHT Educational Telecommunications (WMHT), Q-TV (WDCQ), WTVS Detroit Public TV (WTVS), CMU Public Television (WCMU), WKAR-TV (WKAR), WNMU-TV Public TV 13 (WNMU), WDSE - WRPT (WDSE), WGTE TV (WGTE), Lakeland Public Television (KAWE), KMOS-TV - Channels 6.1, 6.2 and 6.3 (KMOS), MontanaPBS (KUSM), KRWG/Channel 22 (KRWG), KACV (KACV), KCOS/Channel 13 (KCOS), WCNY/Channel 24 (WCNY), WNED (WNED), WPBS (WPBS), WSKG Public TV (WSKG), WXXI (WXXI), WPSU (WPSU), WVIA Public Media Studios (WVIA), WTVI (WTVI), Western Reserve PBS (WNEO), WVIZ/PBS ideastream (WVIZ), KCTS 9 (KCTS), Basin PBS (KPBT), KUHT / Channel 8 (KUHT), KLRN (KLRN), KLRU (KLRU), WTJX Channel 12 (WTJX), WCVE PBS (WCVE), KBTC Public Television (KBTC)
 - **PearVideo**
@@ -748,6 +752,7 @@
 - **play.fm**
 - **player.sky.it**
 - **PlayPlusTV**
+ - **PlayStuff**
 - **PlaysTV**
 - **Playtvak**: Playtvak.cz, iDNES.cz and Lidovky.cz
 - **Playvid**
@@ -856,6 +861,7 @@
 - **safari**: safaribooksonline.com online video
 - **safari:api**
 - **safari:course**: safaribooksonline.com online courses
+ - **Saitosan**
 - **SAKTV**
 - **SaltTV**
 - **SampleFocus**
@@ -880,6 +886,7 @@
 - **Shahid**
 - **ShahidShow**
 - **Shared**: shared.sx
+ - **ShemarooMe**
 - **ShowRoomLive**
 - **simplecast**
 - **simplecast:episode**
@@ -899,6 +906,7 @@
 - **Snotr**
 - **Sohu**
 - **SonyLIV**
+ - **SonyLIVSeries**
 - **soundcloud**
 - **soundcloud:playlist**
 - **soundcloud:search**: Soundcloud search
@@ -977,6 +985,7 @@
 - **Telecinco**: telecinco.es, cuatro.com and mediaset.es
 - **Telegraaf**
 - **TeleMB**
+ - **Telemundo**
 - **TeleQuebec**
 - **TeleQuebecEmission**
 - **TeleQuebecLive**
@@ -1160,6 +1169,7 @@
 - **VODPlatform**
 - **VoiceRepublic**
 - **Voot**
+ - **VootSeries**
 - **VoxMedia**
 - **VoxMediaVolume**
 - **vpro**: npo.nl, ntr.nl, omroepwnl.nl, zapp.nl and npo3.nl
@@ -1189,6 +1199,7 @@
 - **wdr:mobile**
 - **WDRElefant**
 - **WDRPage**
+ - **web.archive:youtube**: web.archive.org saved youtube videos
 - **Webcaster**
 - **WebcasterFeed**
 - **WebOfStories**
@@ -1,41 +1,40 @@
 {
-    "consoletitle": false, 
-    "continuedl": true, 
-    "forcedescription": false, 
-    "forcefilename": false, 
-    "forceformat": false, 
-    "forcethumbnail": false, 
-    "forcetitle": false, 
-    "forceurl": false, 
+    "consoletitle": false,
+    "continuedl": true,
+    "forcedescription": false,
+    "forcefilename": false,
+    "forceformat": false,
+    "forcethumbnail": false,
+    "forcetitle": false,
+    "forceurl": false,
    "force_write_download_archive": false,
    "format": "best",
-    "ignoreerrors": false, 
-    "listformats": null, 
-    "logtostderr": false, 
-    "matchtitle": null, 
-    "max_downloads": null, 
-    "overwrites": null, 
-    "nopart": false, 
-    "noprogress": false, 
-    "outtmpl": "%(id)s.%(ext)s", 
-    "password": null, 
-    "playlistend": -1, 
-    "playliststart": 1, 
-    "prefer_free_formats": false, 
-    "quiet": false, 
-    "ratelimit": null, 
-    "rejecttitle": null, 
-    "retries": 10, 
-    "simulate": false, 
-    "subtitleslang": null, 
+    "ignoreerrors": false,
+    "listformats": null,
+    "logtostderr": false,
+    "matchtitle": null,
+    "max_downloads": null,
+    "overwrites": null,
+    "nopart": false,
+    "noprogress": false,
+    "outtmpl": "%(id)s.%(ext)s",
+    "password": null,
+    "playliststart": 1,
+    "prefer_free_formats": false,
+    "quiet": false,
+    "ratelimit": null,
+    "rejecttitle": null,
+    "retries": 10,
+    "simulate": false,
+    "subtitleslang": null,
    "subtitlesformat": "best",
-    "test": true, 
-    "updatetime": true, 
-    "usenetrc": false, 
-    "username": null, 
-    "verbose": true, 
-    "writedescription": false, 
-    "writeinfojson": true, 
+    "test": true,
+    "updatetime": true,
+    "usenetrc": false,
+    "username": null,
+    "verbose": true,
+    "writedescription": false,
+    "writeinfojson": true,
    "writeannotations": false,
    "writelink": false,
    "writeurllink": false,
@@ -29,6 +29,7 @@ class YDL(FakeYDL):
        self.msgs = []

    def process_info(self, info_dict):
+        info_dict.pop('__original_infodict', None)
        self.downloaded_info_dicts.append(info_dict)

    def to_screen(self, msg):
@@ -755,7 +756,7 @@ class TestYoutubeDL(unittest.TestCase):
            def process_info(self, info_dict):
                super(YDL, self).process_info(info_dict)

-            def _match_entry(self, info_dict, incomplete):
+            def _match_entry(self, info_dict, incomplete=False):
                res = super(FilterYDL, self)._match_entry(info_dict, incomplete)
                if res is None:
                    self.downloaded_info_dicts.append(info_dict)
@@ -121,6 +121,7 @@ def generator(test_case, tname):
        params['outtmpl'] = tname + '_' + params['outtmpl']
        if is_playlist and 'playlist' not in test_case:
            params.setdefault('extract_flat', 'in_playlist')
+            params.setdefault('playlistend', test_case.get('playlist_mincount'))
            params.setdefault('skip_download', True)

        ydl = YoutubeDL(params, auto_init=False)
@@ -8,7 +8,11 @@ import sys
 import unittest
 sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))

-from yt_dlp.postprocessor import MetadataFromFieldPP, MetadataFromTitlePP
+from yt_dlp.postprocessor import (
+    FFmpegThumbnailsConvertorPP,
+    MetadataFromFieldPP,
+    MetadataFromTitlePP,
+)


 class TestMetadataFromField(unittest.TestCase):
@@ -30,3 +34,24 @@ class TestMetadataFromTitle(unittest.TestCase):
    def test_format_to_regex(self):
        pp = MetadataFromTitlePP(None, '%(title)s - %(artist)s')
        self.assertEqual(pp._titleregex, r'(?P<title>.+)\ \-\ (?P<artist>.+)')
+
+
+class TestConvertThumbnail(unittest.TestCase):
+    def test_escaping(self):
+        pp = FFmpegThumbnailsConvertorPP()
+        if not pp.available:
+            print('Skipping: ffmpeg not found')
+            return
+
+        file = 'test/testdata/thumbnails/foo %d bar/foo_%d.{}'
+        tests = (('webp', 'png'), ('png', 'jpg'))
+
+        for inp, out in tests:
+            out_file = file.format(out)
+            if os.path.exists(out_file):
+                os.remove(out_file)
+            pp.convert_thumbnail(file.format(inp), out)
+            assert os.path.exists(out_file)
+
+        for _, out in tests:
+            os.remove(file.format(out))
@@ -66,6 +66,7 @@ from yt_dlp.utils import (
    sanitize_filename,
    sanitize_path,
    sanitize_url,
+    sanitized_Request,
    expand_path,
    prepend_extension,
    replace_extension,
@@ -238,6 +239,16 @@ class TestUtil(unittest.TestCase):
        self.assertEqual(sanitize_url('httpss://foo.bar'), 'https://foo.bar')
        self.assertEqual(sanitize_url('rmtps://foo.bar'), 'rtmps://foo.bar')
        self.assertEqual(sanitize_url('https://foo.bar'), 'https://foo.bar')
+        self.assertEqual(sanitize_url('foo bar'), 'foo bar')
+
+    def test_extract_basic_auth(self):
+        auth_header = lambda url: sanitized_Request(url).get_header('Authorization')
+        self.assertFalse(auth_header('http://foo.bar'))
+        self.assertFalse(auth_header('http://:foo.bar'))
+        self.assertEqual(auth_header('http://@foo.bar'), 'Basic Og==')
+        self.assertEqual(auth_header('http://:pass@foo.bar'), 'Basic OnBhc3M=')
+        self.assertEqual(auth_header('http://user:@foo.bar'), 'Basic dXNlcjo=')
+        self.assertEqual(auth_header('http://user:pass@foo.bar'), 'Basic dXNlcjpwYXNz')

    def test_expand_path(self):
        def env(var):
@@ -48,7 +48,6 @@ from .utils import (
    date_from_str,
    DateRange,
    DEFAULT_OUTTMPL,
-    OUTTMPL_TYPES,
    determine_ext,
    determine_protocol,
    DOT_DESKTOP_LINK_TEMPLATE,
@@ -57,8 +56,8 @@ from .utils import (
    DownloadError,
    encode_compat_str,
    encodeFilename,
-    error_to_compat_str,
    EntryNotInPlaylist,
+    error_to_compat_str,
    ExistingVideoReached,
    expand_path,
    ExtractorError,
@@ -71,12 +70,14 @@ from .utils import (
    int_or_none,
    iri_to_uri,
    ISO3166Utils,
+    LazyList,
    locked_file,
    make_dir,
    make_HTTPS_handler,
    MaxDownloadsReached,
    network_exceptions,
    orderedSet,
+    OUTTMPL_TYPES,
    PagedList,
    parse_filesize,
    PerRequestProxyHandler,
@@ -84,11 +85,12 @@ from .utils import (
    PostProcessingError,
    preferredencoding,
    prepend_extension,
+    process_communicate_or_kill,
    random_uuidv4,
    register_socks_protocols,
+    RejectedVideoReached,
    render_table,
    replace_extension,
-    RejectedVideoReached,
    SameFileError,
    sanitize_filename,
    sanitize_path,
@@ -109,7 +111,6 @@ from .utils import (
    YoutubeDLCookieProcessor,
    YoutubeDLHandler,
    YoutubeDLRedirectHandler,
-    process_communicate_or_kill,
 )
 from .cache import Cache
 from .extractor import (
@@ -177,13 +178,14 @@ class YoutubeDL(object):
    verbose:           Print additional info to stdout.
    quiet:             Do not print messages to stdout.
    no_warnings:       Do not print out anything for warnings.
-    forceurl:          Force printing final URL.
-    forcetitle:        Force printing title.
-    forceid:           Force printing ID.
-    forcethumbnail:    Force printing thumbnail URL.
-    forcedescription:  Force printing description.
-    forcefilename:     Force printing final filename.
-    forceduration:     Force printing duration.
+    forceprint:        A list of templates to force print
+    forceurl:          Force printing final URL. (Deprecated)
+    forcetitle:        Force printing title. (Deprecated)
+    forceid:           Force printing ID. (Deprecated)
+    forcethumbnail:    Force printing thumbnail URL. (Deprecated)
+    forcedescription:  Force printing description. (Deprecated)
+    forcefilename:     Force printing final filename. (Deprecated)
+    forceduration:     Force printing duration. (Deprecated)
    forcejson:         Force printing info_dict as JSON.
    dump_single_json:  Force printing the info_dict of the whole playlist
                       (or video) as a single JSON line.
@@ -387,7 +389,8 @@ class YoutubeDL(object):
                       use downloader suggested by extractor if None.
    compat_opts:       Compatibility options. See "Differences in default behavior".
                       Note that only format-sort, format-spec, no-live-chat,
-                       playlist-index, list-formats, no-youtube-channel-redirect
+                       no-attach-info-json, playlist-index, list-formats,
+                       no-direct-merge, no-youtube-channel-redirect,
                       and no-youtube-unavailable-videos works when used via the API

    The following parameters are not used by YoutubeDL itself, they are used by
@@ -542,8 +545,7 @@ class YoutubeDL(object):
        def preload_download_archive(fn):
            if fn is None:
                return False
-            if self.params.get('verbose'):
-                self._write_string('[debug] Loading archive file %r\n' % fn)
+            self.write_debug('Loading archive file %r\n' % fn)
            try:
                with locked_file(fn, 'r', encoding='utf-8') as archive_file:
                    for line in archive_file:
@@ -649,35 +651,25 @@ class YoutubeDL(object):
                      for _ in range(line_count))
        return res[:-len('\n')]

-    def to_screen(self, message, skip_eol=False):
-        """Print message to stdout if not in quiet mode."""
-        return self.to_stdout(
-            message, skip_eol,
-            quiet=self.params.get('quiet', False))
-
    def _write_string(self, s, out=None):
        write_string(s, out=out, encoding=self.params.get('encoding'))

    def to_stdout(self, message, skip_eol=False, quiet=False):
-        """Print message to stdout if not in quiet mode."""
+        """Print message to stdout"""
        if self.params.get('logger'):
            self.params['logger'].debug(message)
-        elif not quiet:
-            message = self._bidi_workaround(message)
-            terminator = ['\n', ''][skip_eol]
-            output = message + terminator
-
-            self._write_string(output, self._screen_file)
+        elif not quiet or self.params.get('verbose'):
+            self._write_string(
+                '%s%s' % (self._bidi_workaround(message), ('' if skip_eol else '\n')),
+                self._err_file if quiet else self._screen_file)

    def to_stderr(self, message):
-        """Print message to stderr."""
+        """Print message to stderr"""
        assert isinstance(message, compat_str)
        if self.params.get('logger'):
            self.params['logger'].error(message)
        else:
-            message = self._bidi_workaround(message)
-            output = message + '\n'
-            self._write_string(output, self._err_file)
+            self._write_string('%s\n' % self._bidi_workaround(message), self._err_file)

    def to_console_title(self, message):
        if not self.params.get('consoletitle', False):
@@ -739,7 +731,8 @@ class YoutubeDL(object):
                else:
                    tb_data = traceback.format_list(traceback.extract_stack())
                    tb = ''.join(tb_data)
-            self.to_stderr(tb)
+            if tb:
+                self.to_stderr(tb)
        if not self.params.get('ignoreerrors', False):
            if sys.exc_info()[0] and hasattr(sys.exc_info()[1], 'exc_info') and sys.exc_info()[1].exc_info[0]:
                exc_info = sys.exc_info()[1].exc_info
@@ -748,6 +741,11 @@ class YoutubeDL(object):
            raise DownloadError(message, exc_info)
        self._download_retcode = 1

+    def to_screen(self, message, skip_eol=False):
+        """Print message to stdout if not in quiet mode"""
+        self.to_stdout(
+            message, skip_eol, quiet=self.params.get('quiet', False))
+
    def report_warning(self, message):
        '''
        Print the message to stderr, it will be prefixed with 'WARNING:'
@@ -777,6 +775,16 @@ class YoutubeDL(object):
        error_message = '%s %s' % (_msg_header, message)
        self.trouble(error_message, tb)

+    def write_debug(self, message):
+        '''Log debug message or Print message to stderr'''
+        if not self.params.get('verbose', False):
+            return
+        message = '[debug] %s' % message
+        if self.params.get('logger'):
+            self.params['logger'].debug(message)
+        else:
+            self._write_string('%s\n' % message)
+
    def report_file_already_downloaded(self, file_name):
        """Report file has already been fully downloaded."""
        try:
@@ -812,7 +820,7 @@ class YoutubeDL(object):

        # duration_string
        template_dict['duration_string'] = (  # %(duration>%H-%M-%S)s is wrong if duration > 24hrs
-            formatSeconds(info_dict['duration'], '-')
+            formatSeconds(info_dict['duration'], '-' if sanitize else ':')
            if info_dict.get('duration', None) is not None
            else None)

@@ -1005,11 +1013,12 @@ class YoutubeDL(object):
            path = encodeFilename(path, True).decode(preferredencoding())
        return sanitize_path(path, force=self.params.get('windowsfilenames'))

-    def _match_entry(self, info_dict, incomplete):
+    def _match_entry(self, info_dict, incomplete=False, silent=False):
        """ Returns None if the file should be downloaded """

+        video_title = info_dict.get('title', info_dict.get('id', 'video'))
+
        def check_filter():
-            video_title = info_dict.get('title', info_dict.get('id', 'video'))
            if 'title' in info_dict:
                # This can happen when we're just evaluating the playlist
                title = info_dict['title']
@@ -1036,8 +1045,6 @@ class YoutubeDL(object):
                    return 'Skipping %s, because it has exceeded the maximum view count (%d/%d)' % (video_title, view_count, max_views)
            if age_restricted(info_dict.get('age_limit'), self.params.get('age_limit')):
                return 'Skipping "%s" because it is age restricted' % video_title
-            if self.in_download_archive(info_dict):
-                return '%s has already been recorded in archive' % video_title

            if not incomplete:
                match_filter = self.params.get('match_filter')
@@ -1047,13 +1054,17 @@ class YoutubeDL(object):
                        return ret
            return None

-        reason = check_filter()
+        if self.in_download_archive(info_dict):
+            reason = '%s has already been recorded in the archive' % video_title
+            break_opt, break_err = 'break_on_existing', ExistingVideoReached
+        else:
+            reason = check_filter()
+            break_opt, break_err = 'break_on_reject', RejectedVideoReached
        if reason is not None:
-            self.to_screen('[download] ' + reason)
-            if reason.endswith('has already been recorded in the archive') and self.params.get('break_on_existing', False):
-                raise ExistingVideoReached()
-            elif self.params.get('break_on_reject', False):
-                raise RejectedVideoReached()
+            if not silent:
+                self.to_screen('[download] ' + reason)
+            if self.params.get(break_opt, False):
+                raise break_err()
        return reason

    @staticmethod
@@ -1178,14 +1189,29 @@ class YoutubeDL(object):

        if result_type == 'video':
            self.add_extra_info(ie_result, extra_info)
-            return self.process_video_result(ie_result, download=download)
+            ie_result = self.process_video_result(ie_result, download=download)
+            additional_urls = (ie_result or {}).get('additional_urls')
+            if additional_urls:
+                # TODO: Improve MetadataFromFieldPP to allow setting a list
+                if isinstance(additional_urls, compat_str):
+                    additional_urls = [additional_urls]
+                self.to_screen(
+                    '[info] %s: %d additional URL(s) requested' % (ie_result['id'], len(additional_urls)))
+                self.write_debug('Additional URLs: "%s"' % '", "'.join(additional_urls))
+                ie_result['additional_entries'] = [
+                    self.extract_info(
+                        url, download, extra_info,
+                        force_generic_extractor=self.params.get('force_generic_extractor'))
+                    for url in additional_urls
+                ]
+            return ie_result
        elif result_type == 'url':
            # We have to add extra_info to the results because it may be
            # contained in a playlist
-            return self.extract_info(ie_result['url'],
-                                     download,
-                                     ie_key=ie_result.get('ie_key'),
-                                     extra_info=extra_info)
+            return self.extract_info(
+                ie_result['url'], download,
+                ie_key=ie_result.get('ie_key'),
+                extra_info=extra_info)
        elif result_type == 'url_transparent':
            # Use the information from the embedding page
            info = self.extract_info(
@@ -1229,6 +1255,7 @@ class YoutubeDL(object):

            self._playlist_level += 1
            self._playlist_urls.add(webpage_url)
+            self._sanitize_thumbnails(ie_result)
            try:
                return self.__process_playlist(ie_result, download)
            finally:
@@ -1280,7 +1307,7 @@ class YoutubeDL(object):

        playlist_results = []

-        playliststart = self.params.get('playliststart', 1) - 1
+        playliststart = self.params.get('playliststart', 1)
        playlistend = self.params.get('playlistend')
        # For backwards compatibility, interpret -1 as whole list
        if playlistend == -1:
@@ -1300,50 +1327,43 @@ class YoutubeDL(object):
            playlistitems = orderedSet(iter_playlistitems(playlistitems_str))

        ie_entries = ie_result['entries']
+        msg = (
+            'Downloading %d videos' if not isinstance(ie_entries, list)
+            else 'Collected %d videos; downloading %%d of them' % len(ie_entries))
+        if not isinstance(ie_entries, (list, PagedList)):
+            ie_entries = LazyList(ie_entries)

-        def make_playlistitems_entries(list_ie_entries):
-            num_entries = len(list_ie_entries)
-            for i in playlistitems:
-                if -num_entries < i <= num_entries:
-                    yield list_ie_entries[i - 1]
-                elif incomplete_entries:
+        entries = []
+        for i in playlistitems or itertools.count(playliststart):
+            if playlistitems is None and playlistend is not None and playlistend < i:
+                break
+            entry = None
+            try:
+                entry = ie_entries[i - 1]
+                if entry is None:
                    raise EntryNotInPlaylist()
-
-        if isinstance(ie_entries, list):
-            n_all_entries = len(ie_entries)
-            if playlistitems:
-                entries = list(make_playlistitems_entries(ie_entries))
-            else:
-                entries = ie_entries[playliststart:playlistend]
-            n_entries = len(entries)
-            msg = 'Collected %d videos; downloading %d of them' % (n_all_entries, n_entries)
-        elif isinstance(ie_entries, PagedList):
-            if playlistitems:
-                entries = []
-                for item in playlistitems:
-                    entries.extend(ie_entries.getslice(
-                        item - 1, item
-                    ))
-            else:
-                entries = ie_entries.getslice(
-                    playliststart, playlistend)
-            n_entries = len(entries)
-            msg = 'Downloading %d videos' % n_entries
-        else:  # iterable
-            if playlistitems:
-                entries = list(make_playlistitems_entries(list(itertools.islice(
-                    ie_entries, 0, max(playlistitems)))))
-            else:
-                entries = list(itertools.islice(
-                    ie_entries, playliststart, playlistend))
-            n_entries = len(entries)
-            msg = 'Downloading %d videos' % n_entries
-
-        if any((entry is None for entry in entries)):
-            raise EntryNotInPlaylist()
-        if not playlistitems and (playliststart or playlistend):
-            playlistitems = list(range(1 + playliststart, 1 + playliststart + len(entries)))
+            except (IndexError, EntryNotInPlaylist):
+                if incomplete_entries:
+                    raise EntryNotInPlaylist()
+                elif not playlistitems:
+                    break
+            entries.append(entry)
+            try:
+                if entry is not None:
+                    self._match_entry(entry, incomplete=True, silent=True)
+            except (ExistingVideoReached, RejectedVideoReached):
+                break
        ie_result['entries'] = entries
+
+        # Save playlist_index before re-ordering
+        entries = [
+            ((playlistitems[i - 1] if playlistitems else i), entry)
+            for i, entry in enumerate(entries, 1)
+            if entry is not None]
+        n_entries = len(entries)
+
+        if not playlistitems and (playliststart or playlistend):
+            playlistitems = list(range(playliststart, playliststart + n_entries))
        ie_result['requested_entries'] = playlistitems

        if self.params.get('allow_playlist_files', True):
@@ -1370,6 +1390,9 @@ class YoutubeDL(object):
                    except (OSError, IOError):
                        self.report_error('Cannot write playlist metadata to JSON file ' + infofn)

+            # TODO: This should be passed to ThumbnailsConvertor if necessary
+            self._write_thumbnails(ie_copy, self.prepare_filename(ie_copy, 'pl_thumbnail'))
+
            if self.params.get('writedescription', False):
                descfn = self.prepare_filename(ie_copy, 'pl_description')
                if not self._ensure_dir_exists(encodeFilename(descfn)):
@@ -1387,11 +1410,6 @@ class YoutubeDL(object):
                        self.report_error('Cannot write playlist description file ' + descfn)
                        return

-        # Save playlist_index before re-ordering
-        entries = [
-            ((playlistitems[i - 1] if playlistitems else i), entry)
-            for i, entry in enumerate(entries, 1)]
-
        if self.params.get('playlistreverse', False):
            entries = entries[::-1]
        if self.params.get('playlistrandom', False):
@@ -1399,7 +1417,7 @@ class YoutubeDL(object):

        x_forwarded_for = ie_result.get('__x_forwarded_for_ip')

-        self.to_screen('[%s] playlist %s: %s' % (ie_result['extractor'], playlist, msg))
+        self.to_screen('[%s] playlist %s: %s' % (ie_result['extractor'], playlist, msg % n_entries))
        failures = 0
        max_failures = self.params.get('skip_playlist_after_errors') or float('inf')
        for i, entry_tuple in enumerate(entries, 1):
@@ -1717,9 +1735,13 @@ class YoutubeDL(object):
                    expand_path(paths.get('home', '').strip()),
                    expand_path(paths.get('temp', '').strip()),
                    'ytdl.%s.f%s.check-format' % (random_uuidv4(), f['format_id']))
-                dl, _ = self.dl(temp_file, f, test=True)
-                if os.path.exists(temp_file):
-                    os.remove(temp_file)
+                try:
+                    dl, _ = self.dl(temp_file, f, test=True)
+                except (ExtractorError, IOError, OSError, ValueError) + network_exceptions:
+                    dl = False
+                finally:
+                    if os.path.exists(temp_file):
+                        os.remove(temp_file)
                if dl:
                    yield f
                else:
@@ -1882,6 +1904,27 @@ class YoutubeDL(object):
        self.cookiejar.add_cookie_header(pr)
        return pr.get_header('Cookie')

+    @staticmethod
+    def _sanitize_thumbnails(info_dict):
+        thumbnails = info_dict.get('thumbnails')
+        if thumbnails is None:
+            thumbnail = info_dict.get('thumbnail')
+            if thumbnail:
+                info_dict['thumbnails'] = thumbnails = [{'url': thumbnail}]
+        if thumbnails:
+            thumbnails.sort(key=lambda t: (
+                t.get('preference') if t.get('preference') is not None else -1,
+                t.get('width') if t.get('width') is not None else -1,
+                t.get('height') if t.get('height') is not None else -1,
+                t.get('id') if t.get('id') is not None else '',
+                t.get('url')))
+            for i, t in enumerate(thumbnails):
+                t['url'] = sanitize_url(t['url'])
+                if t.get('width') and t.get('height'):
+                    t['resolution'] = '%dx%d' % (t['width'], t['height'])
+                if t.get('id') is None:
+                    t['id'] = '%d' % i
+
    def process_video_result(self, info_dict, download=True):
        assert info_dict.get('_type', 'video') == 'video'

@@ -1918,29 +1961,14 @@ class YoutubeDL(object):
            info_dict['playlist'] = None
            info_dict['playlist_index'] = None

-        thumbnails = info_dict.get('thumbnails')
-        if thumbnails is None:
-            thumbnail = info_dict.get('thumbnail')
-            if thumbnail:
-                info_dict['thumbnails'] = thumbnails = [{'url': thumbnail}]
-        if thumbnails:
-            thumbnails.sort(key=lambda t: (
-                t.get('preference') if t.get('preference') is not None else -1,
-                t.get('width') if t.get('width') is not None else -1,
-                t.get('height') if t.get('height') is not None else -1,
-                t.get('id') if t.get('id') is not None else '', t.get('url')))
-            for i, t in enumerate(thumbnails):
-                t['url'] = sanitize_url(t['url'])
-                if t.get('width') and t.get('height'):
-                    t['resolution'] = '%dx%d' % (t['width'], t['height'])
-                if t.get('id') is None:
-                    t['id'] = '%d' % i
+        self._sanitize_thumbnails(info_dict)

        if self.params.get('list_thumbnails'):
            self.list_thumbnails(info_dict)
            return

        thumbnail = info_dict.get('thumbnail')
+        thumbnails = info_dict.get('thumbnails')
        if thumbnail:
            info_dict['thumbnail'] = sanitize_url(thumbnail)
        elif thumbnails:
@@ -2072,6 +2100,9 @@ class YoutubeDL(object):
            # element in the 'formats' field in info_dict is info_dict itself,
            # which can't be exported to json
            info_dict['formats'] = formats
+
+        info_dict, _ = self.pre_process(info_dict)
+
        if self.params.get('listformats'):
            if not info_dict.get('formats'):
                raise ExtractorError('No video formats found', expected=True)
@@ -2081,8 +2112,7 @@ class YoutubeDL(object):
        req_format = self.params.get('format')
        if req_format is None:
            req_format = self._default_format_spec(info_dict, download=download)
-            if self.params.get('verbose'):
-                self.to_screen('[debug] Default format spec: %s' % req_format)
+            self.write_debug('Default format spec: %s' % req_format)

        format_selector = self.build_format_selector(req_format)

@@ -2120,14 +2150,13 @@ class YoutubeDL(object):
                self.report_warning('Requested format is not available')
        elif download:
            self.to_screen(
-                '[info] %s: Downloading format(s) %s'
-                % (info_dict['id'], ", ".join([f['format_id'] for f in formats_to_download])))
-            if len(formats_to_download) > 1:
-                self.to_screen(
-                    '[info] %s: Downloading video in %s formats'
-                    % (info_dict['id'], len(formats_to_download)))
+                '[info] %s: Downloading %d format(s): %s' % (
+                    info_dict['id'], len(formats_to_download),
+                    ", ".join([f['format_id'] for f in formats_to_download])))
            for fmt in formats_to_download:
                new_info = dict(info_dict)
+                # Save a reference to the original info_dict so that it can be modified in process_info if needed
+                new_info['__original_infodict'] = info_dict
                new_info.update(fmt)
                self.process_info(new_info)
        # We update the info dict with the best quality format (backwards compatibility)
@@ -2172,6 +2201,7 @@ class YoutubeDL(object):
            requested_langs = ['en']
        else:
            requested_langs = [list(all_sub_langs)[0]]
+        self.write_debug('Downloading subtitles: %s' % ', '.join(requested_langs))

        formats_query = self.params.get('subtitlesformat', 'best')
        formats_preference = formats_query.split('/') if formats_query else []
@@ -2198,32 +2228,43 @@ class YoutubeDL(object):
        return subs

    def __forced_printings(self, info_dict, filename, incomplete):
-        def print_mandatory(field):
+        def print_mandatory(field, actual_field=None):
+            if actual_field is None:
+                actual_field = field
            if (self.params.get('force%s' % field, False)
-                    and (not incomplete or info_dict.get(field) is not None)):
-                self.to_stdout(info_dict[field])
+                    and (not incomplete or info_dict.get(actual_field) is not None)):
+                self.to_stdout(info_dict[actual_field])

        def print_optional(field):
            if (self.params.get('force%s' % field, False)
                    and info_dict.get(field) is not None):
                self.to_stdout(info_dict[field])

+        info_dict = info_dict.copy()
+        if filename is not None:
+            info_dict['filename'] = filename
+        if info_dict.get('requested_formats') is not None:
+            # For RTMP URLs, also include the playpath
+            info_dict['urls'] = '\n'.join(f['url'] + f.get('play_path', '') for f in info_dict['requested_formats'])
+        elif 'url' in info_dict:
+            info_dict['urls'] = info_dict['url'] + info_dict.get('play_path', '')
+
+        for tmpl in self.params.get('forceprint', []):
+            if re.match(r'\w+$', tmpl):
+                tmpl = '%({})s'.format(tmpl)
+            tmpl, info_copy = self.prepare_outtmpl(tmpl, info_dict)
+            self.to_stdout(tmpl % info_copy)
+
        print_mandatory('title')
        print_mandatory('id')
-        if self.params.get('forceurl', False) and not incomplete:
-            if info_dict.get('requested_formats') is not None:
-                for f in info_dict['requested_formats']:
-                    self.to_stdout(f['url'] + f.get('play_path', ''))
-            else:
-                # For RTMP URLs, also include the playpath
-                self.to_stdout(info_dict['url'] + info_dict.get('play_path', ''))
+        print_mandatory('url', 'urls')
        print_optional('thumbnail')
        print_optional('description')
-        if self.params.get('forcefilename', False) and filename is not None:
-            self.to_stdout(filename)
+        print_optional('filename')
        if self.params.get('forceduration', False) and info_dict.get('duration') is not None:
            self.to_stdout(formatSeconds(info_dict['duration']))
        print_mandatory('format')
+
        if self.params.get('forcejson', False):
            self.post_extract(info_dict)
            self.to_stdout(json.dumps(info_dict, default=repr))
@@ -2249,8 +2290,8 @@ class YoutubeDL(object):
        if not test:
            for ph in self._progress_hooks:
                fd.add_progress_hook(ph)
-            if self.params.get('verbose'):
-                self.to_screen('[debug] Invoking downloader on %r' % info.get('url'))
+            urls = '", "'.join([f['url'] for f in info.get('requested_formats', [])] or [info['url']])
+            self.write_debug('Invoking downloader on "%s"' % urls)
        new_info = dict(info)
        if new_info.get('http_headers') is None:
            new_info['http_headers'] = self._calc_headers(new_info)
@@ -2274,14 +2315,12 @@ class YoutubeDL(object):
        if 'format' not in info_dict:
            info_dict['format'] = info_dict['ext']

-        if self._match_entry(info_dict, incomplete=False) is not None:
+        if self._match_entry(info_dict) is not None:
            return

        self.post_extract(info_dict)
        self._num_downloads += 1

-        info_dict, _ = self.pre_process(info_dict)
-
        # info_dict['_filename'] needs to be set for backward compatibility
        info_dict['_filename'] = full_filename = self.prepare_filename(info_dict, warn=True)
        temp_filename = self.prepare_filename(info_dict, 'temp')
@@ -2376,7 +2415,7 @@ class YoutubeDL(object):
                            self.dl(sub_filename, sub_info.copy(), subtitle=True)
                            sub_info['filepath'] = sub_filename
                            files_to_move[sub_filename] = sub_filename_final
-                        except tuple([ExtractorError, IOError, OSError, ValueError] + network_exceptions) as err:
+                        except (ExtractorError, IOError, OSError, ValueError) + network_exceptions as err:
                            self.report_warning('Unable to download subtitle for "%s": %s' %
                                                (sub_lang, error_to_compat_str(err)))
                            continue
@@ -2491,17 +2530,6 @@ class YoutubeDL(object):

                success = True
                if info_dict.get('requested_formats') is not None:
-                    downloaded = []
-                    merger = FFmpegMergerPP(self)
-                    if self.params.get('allow_unplayable_formats'):
-                        self.report_warning(
-                            'You have requested merging of multiple formats '
-                            'while also allowing unplayable formats to be downloaded. '
-                            'The formats won\'t be merged to prevent data corruption.')
-                    elif not merger.available:
-                        self.report_warning(
-                            'You have requested merging of multiple formats but ffmpeg is not installed. '
-                            'The formats won\'t be merged.')

                    def compatible_formats(formats):
                        # TODO: some formats actually allow this (mkv, webm, ogg, mp4), but not all of them.
@@ -2549,27 +2577,57 @@ class YoutubeDL(object):
                    temp_filename = correct_ext(temp_filename)
                    dl_filename = existing_file(full_filename, temp_filename)
                    info_dict['__real_download'] = False
-                    if dl_filename is None:
-                        for f in requested_formats:
-                            new_info = dict(info_dict)
-                            new_info.update(f)
-                            fname = prepend_extension(
-                                self.prepare_filename(new_info, 'temp'),
-                                'f%s' % f['format_id'], new_info['ext'])
-                            if not self._ensure_dir_exists(fname):
-                                return
-                            downloaded.append(fname)
-                            partial_success, real_download = self.dl(fname, new_info)
-                            info_dict['__real_download'] = info_dict['__real_download'] or real_download
-                            success = success and partial_success
-                        if merger.available and not self.params.get('allow_unplayable_formats'):
-                            info_dict['__postprocessors'].append(merger)
-                            info_dict['__files_to_merge'] = downloaded
-                            # Even if there were no downloads, it is being merged only now
-                            info_dict['__real_download'] = True
-                        else:
-                            for file in downloaded:
-                                files_to_move[file] = None
+
+                    _protocols = set(determine_protocol(f) for f in requested_formats)
+                    if len(_protocols) == 1:
+                        info_dict['protocol'] = _protocols.pop()
+                    directly_mergable = (
+                        'no-direct-merge' not in self.params.get('compat_opts', [])
+                        and info_dict.get('protocol') is not None  # All requested formats have same protocol
+                        and not self.params.get('allow_unplayable_formats')
+                        and get_suitable_downloader(info_dict, self.params).__name__ == 'FFmpegFD')
+                    if directly_mergable:
+                        info_dict['url'] = requested_formats[0]['url']
+                        # Treat it as a single download
+                        dl_filename = existing_file(full_filename, temp_filename)
+                        if dl_filename is None:
+                            success, real_download = self.dl(temp_filename, info_dict)
+                            info_dict['__real_download'] = real_download
+                    else:
+                        downloaded = []
+                        merger = FFmpegMergerPP(self)
+                        if self.params.get('allow_unplayable_formats'):
+                            self.report_warning(
+                                'You have requested merging of multiple formats '
+                                'while also allowing unplayable formats to be downloaded. '
+                                'The formats won\'t be merged to prevent data corruption.')
+                        elif not merger.available:
+                            self.report_warning(
+                                'You have requested merging of multiple formats but ffmpeg is not installed. '
+                                'The formats won\'t be merged.')
+
+                        if dl_filename is None:
+                            for f in requested_formats:
+                                new_info = dict(info_dict)
+                                del new_info['requested_formats']
+                                new_info.update(f)
+                                fname = prepend_extension(
+                                    self.prepare_filename(new_info, 'temp'),
+                                    'f%s' % f['format_id'], new_info['ext'])
+                                if not self._ensure_dir_exists(fname):
+                                    return
+                                downloaded.append(fname)
+                                partial_success, real_download = self.dl(fname, new_info)
+                                info_dict['__real_download'] = info_dict['__real_download'] or real_download
+                                success = success and partial_success
+                            if merger.available and not self.params.get('allow_unplayable_formats'):
+                                info_dict['__postprocessors'].append(merger)
+                                info_dict['__files_to_merge'] = downloaded
+                                # Even if there were no downloads, it is being merged only now
+                                info_dict['__real_download'] = True
+                            else:
+                                for file in downloaded:
+                                    files_to_move[file] = None
                else:
                    # Just a single file
                    dl_filename = existing_file(full_filename, temp_filename)
@@ -2719,18 +2777,20 @@ class YoutubeDL(object):

    @staticmethod
    def filter_requested_info(info_dict, actually_filter=True):
-        if not actually_filter:
+        remove_keys = ['__original_infodict']  # Always remove this since this may contain a copy of the entire dict
+        keep_keys = ['_type'],  # Always keep this to facilitate load-info-json
+        if actually_filter:
+            remove_keys += ('requested_formats', 'requested_subtitles', 'requested_entries', 'filepath', 'entries')
+            empty_values = (None, {}, [], set(), tuple())
+            reject = lambda k, v: k not in keep_keys and (
+                k.startswith('_') or k in remove_keys or v in empty_values)
+        else:
            info_dict['epoch'] = int(time.time())
-            return info_dict
-        exceptions = {
-            'remove': ['requested_formats', 'requested_subtitles', 'requested_entries', 'filepath', 'entries'],
-            'keep': ['_type'],
-        }
-        keep_key = lambda k: k in exceptions['keep'] or not (k.startswith('_') or k in exceptions['remove'])
+            reject = lambda k, v: k in remove_keys
        filter_fn = lambda obj: (
-            list(map(filter_fn, obj)) if isinstance(obj, (list, tuple))
+            list(map(filter_fn, obj)) if isinstance(obj, (list, tuple, set))
            else obj if not isinstance(obj, dict)
-            else dict((k, filter_fn(v)) for k, v in obj.items() if keep_key(k)))
+            else dict((k, filter_fn(v)) for k, v in obj.items() if not reject(k, v)))
        return filter_fn(info_dict)

    def run_pp(self, pp, infodict):
@@ -2763,13 +2823,14 @@ class YoutubeDL(object):
                    actual_post_extract(video_dict or {})
                return

-            if '__post_extractor' not in info_dict:
-                return
-            post_extractor = info_dict['__post_extractor']
-            if post_extractor:
-                info_dict.update(post_extractor().items())
-            del info_dict['__post_extractor']
-            return
+            post_extractor = info_dict.get('__post_extractor') or (lambda: {})
+            extra = post_extractor().items()
+            info_dict.update(extra)
+            info_dict.pop('__post_extractor', None)
+
+            original_infodict = info_dict.get('__original_infodict') or {}
+            original_infodict.update(extra)
+            original_infodict.pop('__post_extractor', None)

        actual_post_extract(info_dict or {})

@@ -2982,10 +3043,17 @@ class YoutubeDL(object):
            return
        self.to_screen(
            'Available %s for %s:' % (name, video_id))
+
+        def _row(lang, formats):
+            exts, names = zip(*((f['ext'], f.get('name', 'unknown')) for f in reversed(formats)))
+            if len(set(names)) == 1:
+                names = [] if names[0] == 'unknown' else names[:1]
+            return [lang, ', '.join(names), ', '.join(exts)]
+
        self.to_screen(render_table(
-            ['Language', 'formats'],
-            [[lang, ', '.join(f['ext'] for f in reversed(formats))]
-                for lang, formats in subtitles.items()]))
+            ['Language', 'Name', 'Formats'],
+            [_row(lang, formats) for lang, formats in subtitles.items()],
+            hideEmpty=True))

    def urlopen(self, req):
        """ Start an HTTP download """
@@ -3164,7 +3232,7 @@ class YoutubeDL(object):
            thumb_ext = determine_ext(t['url'], 'jpg')
            suffix = '%s.' % t['id'] if multiple else ''
            thumb_display_id = '%s ' % t['id'] if multiple else ''
-            t['filepath'] = thumb_filename = replace_extension(filename, suffix + thumb_ext, info_dict.get('ext'))
+            thumb_filename = replace_extension(filename, suffix + thumb_ext, info_dict.get('ext'))

            if not self.params.get('overwrites', True) and os.path.exists(encodeFilename(thumb_filename)):
                ret.append(suffix + thumb_ext)
@@ -3180,6 +3248,7 @@ class YoutubeDL(object):
                    ret.append(suffix + thumb_ext)
                    self.to_screen('[%s] %s: Writing thumbnail %sto: %s' %
                                   (info_dict['extractor'], info_dict['id'], thumb_display_id, thumb_filename))
+                    t['filepath'] = thumb_filename
                except network_exceptions as err:
                    self.report_warning('Unable to download thumbnail "%s": %s' %
                                        (t['url'], error_to_compat_str(err)))
@@ -31,20 +31,26 @@ from .utils import (
    preferredencoding,
    read_batch_urls,
    RejectedVideoReached,
-    REMUX_EXTENSIONS,
    render_table,
    SameFileError,
    setproctitle,
    std_headers,
    write_string,
 )
-from .update import update_self
+from .update import run_update
 from .downloader import (
    FileDownloader,
 )
 from .extractor import gen_extractors, list_extractors
 from .extractor.common import InfoExtractor
 from .extractor.adobepass import MSO_INFO
+from .postprocessor.ffmpeg import (
+    FFmpegExtractAudioPP,
+    FFmpegSubtitlesConvertorPP,
+    FFmpegThumbnailsConvertorPP,
+    FFmpegVideoConvertorPP,
+    FFmpegVideoRemuxerPP,
+)
 from .postprocessor.metadatafromfield import MetadataFromFieldPP
 from .YoutubeDL import YoutubeDL

@@ -209,25 +215,25 @@ def _real_main(argv=None):
    if opts.playlistend not in (-1, None) and opts.playlistend < opts.playliststart:
        raise ValueError('Playlist end must be greater than playlist start')
    if opts.extractaudio:
-        if opts.audioformat not in ['best', 'aac', 'flac', 'mp3', 'm4a', 'opus', 'vorbis', 'wav']:
+        if opts.audioformat not in ['best'] + list(FFmpegExtractAudioPP.SUPPORTED_EXTS):
            parser.error('invalid audio format specified')
    if opts.audioquality:
        opts.audioquality = opts.audioquality.strip('k').strip('K')
        if not opts.audioquality.isdigit():
            parser.error('invalid audio quality specified')
    if opts.recodevideo is not None:
-        if opts.recodevideo not in REMUX_EXTENSIONS:
-            parser.error('invalid video recode format specified')
+        opts.recodevideo = opts.recodevideo.replace(' ', '')
+        if not re.match(FFmpegVideoConvertorPP.FORMAT_RE, opts.recodevideo):
+            parser.error('invalid video remux format specified')
    if opts.remuxvideo is not None:
        opts.remuxvideo = opts.remuxvideo.replace(' ', '')
-        remux_regex = r'{0}(?:/{0})*$'.format(r'(?:\w+>)?(?:%s)' % '|'.join(REMUX_EXTENSIONS))
-        if not re.match(remux_regex, opts.remuxvideo):
+        if not re.match(FFmpegVideoRemuxerPP.FORMAT_RE, opts.remuxvideo):
            parser.error('invalid video remux format specified')
    if opts.convertsubtitles is not None:
-        if opts.convertsubtitles not in ('srt', 'vtt', 'ass', 'lrc'):
+        if opts.convertsubtitles not in FFmpegSubtitlesConvertorPP.SUPPORTED_EXTS:
            parser.error('invalid subtitle format specified')
    if opts.convertthumbnails is not None:
-        if opts.convertthumbnails not in ('jpg', ):
+        if opts.convertthumbnails not in FFmpegThumbnailsConvertorPP.SUPPORTED_EXTS:
            parser.error('invalid thumbnail format specified')

    if opts.date is not None:
@@ -258,9 +264,9 @@ def _real_main(argv=None):
        return parsed_compat_opts

    all_compat_opts = [
-        'filename', 'format-sort', 'abort-on-error', 'format-spec', 'multistreams',
-        'no-playlist-metafiles', 'no-live-chat', 'playlist-index', 'list-formats',
-        'no-youtube-channel-redirect', 'no-youtube-unavailable-videos',
+        'filename', 'format-sort', 'abort-on-error', 'format-spec', 'no-playlist-metafiles',
+        'multistreams', 'no-live-chat', 'playlist-index', 'list-formats', 'no-direct-merge',
+        'no-youtube-channel-redirect', 'no-youtube-unavailable-videos', 'no-attach-info-json',
    ]
    compat_opts = parse_compat_opts()

@@ -321,7 +327,7 @@ def _real_main(argv=None):
        if re.match(MetadataFromFieldPP.regex, f) is None:
            parser.error('invalid format string "%s" specified for --parse-metadata' % f)

-    any_getting = opts.geturl or opts.gettitle or opts.getid or opts.getthumbnail or opts.getdescription or opts.getfilename or opts.getformat or opts.getduration or opts.dumpjson or opts.dump_single_json
+    any_getting = opts.forceprint or opts.geturl or opts.gettitle or opts.getid or opts.getthumbnail or opts.getdescription or opts.getfilename or opts.getformat or opts.getduration or opts.dumpjson or opts.dump_single_json
    any_printing = opts.print_json
    download_archive_fn = expand_path(opts.download_archive) if opts.download_archive is not None else opts.download_archive

@@ -480,10 +486,10 @@ def _real_main(argv=None):
        opts.postprocessor_args['default'] = opts.postprocessor_args['default-compat']

    final_ext = (
-        opts.recodevideo
-        or (opts.remuxvideo in REMUX_EXTENSIONS) and opts.remuxvideo
-        or (opts.extractaudio and opts.audioformat != 'best') and opts.audioformat
-        or None)
+        opts.recodevideo if opts.recodevideo in FFmpegVideoConvertorPP.SUPPORTED_EXTS
+        else opts.remuxvideo if opts.remuxvideo in FFmpegVideoRemuxerPP.SUPPORTED_EXTS
+        else opts.audioformat if (opts.extractaudio and opts.audioformat != 'best')
+        else None)

    match_filter = (
        None if opts.match_filter is None
@@ -508,6 +514,7 @@ def _real_main(argv=None):
        'forceduration': opts.getduration,
        'forcefilename': opts.getfilename,
        'forceformat': opts.getformat,
+        'forceprint': opts.forceprint,
        'forcejson': opts.dumpjson or opts.print_json,
        'dump_single_json': opts.dump_single_json,
        'force_write_download_archive': opts.force_write_download_archive,
@@ -656,7 +663,7 @@ def _real_main(argv=None):
        # Update version
        if opts.update_self:
            # If updater returns True, exit. Required for windows
-            if update_self(ydl.to_screen, opts.verbose, ydl._opener):
+            if run_update(ydl):
                if actual_use:
                    sys.exit('ERROR: The program must exit for the update to complete')
                sys.exit()
@@ -164,6 +164,9 @@ class FileDownloader(object):
    def report_error(self, *args, **kargs):
        self.ydl.report_error(*args, **kargs)

+    def write_debug(self, *args, **kargs):
+        self.ydl.write_debug(*args, **kargs)
+
    def slow_down(self, start_time, now, byte_counter):
        """Sleep if the download speed is over the rate limit."""
        rate_limit = self.params.get('ratelimit')
@@ -402,5 +405,4 @@ class FileDownloader(object):
        if exe is None:
            exe = os.path.basename(str_args[0])

-        self.to_screen('[debug] %s command line: %s' % (
-            exe, shell_quote(str_args)))
+        self.write_debug('%s command line: %s' % (exe, shell_quote(str_args)))
@@ -154,8 +154,9 @@ class DashSegmentsFD(FragmentFD):
            max_workers = self.params.get('concurrent_fragment_downloads', 1)
            if can_threaded_download and max_workers > 1:
                self.report_warning('The download speed shown is only of one thread. This is a known issue')
+                _download_fragment = lambda f: (f, download_fragment(f)[1])
                with concurrent.futures.ThreadPoolExecutor(max_workers) as pool:
-                    futures = [pool.submit(download_fragment, fragment) for fragment in fragments_to_download]
+                    futures = [pool.submit(_download_fragment, fragment) for fragment in fragments_to_download]
                    # timeout must be 0 to return instantly
                    done, not_done = concurrent.futures.wait(futures, timeout=0)
                    try:
@@ -169,9 +170,13 @@ class DashSegmentsFD(FragmentFD):
                        # timeout must be none to cancel
                        concurrent.futures.wait(not_done, timeout=None)
                        raise KeyboardInterrupt
-                results = [future.result() for future in futures]

-                for frag_content, frag_index in results:
+                for fragment, frag_index in map(lambda x: x.result(), futures):
+                    fragment_filename = '%s-Frag%d' % (ctx['tmpfilename'], frag_index)
+                    down, frag_sanitized = sanitize_open(fragment_filename, 'rb')
+                    fragment['fragment_filename_sanitized'] = frag_sanitized
+                    frag_content = down.read()
+                    down.close()
                    result = append_fragment(frag_content, frag_index)
                    if not result:
                        return False
@@ -346,7 +346,7 @@ class FFmpegFD(ExternalFD):
        return FFmpegPostProcessor().available

    def _call_downloader(self, tmpfilename, info_dict):
-        url = info_dict['url']
+        urls = [f['url'] for f in info_dict.get('requested_formats', [])] or [info_dict['url']]
        ffpp = FFmpegPostProcessor(downloader=self)
        if not ffpp.available:
            self.report_error('m3u8 download detected but ffmpeg could not be found. Please install')
@@ -378,7 +378,7 @@ class FFmpegFD(ExternalFD):
        # if end_time:
        #     args += ['-t', compat_str(end_time - start_time)]

-        if info_dict.get('http_headers') is not None and re.match(r'^https?://', url):
+        if info_dict.get('http_headers') is not None and re.match(r'^https?://', urls[0]):
            # Trailing \r\n after each HTTP header is important to prevent warning from ffmpeg/avconv:
            # [http @ 00000000003d2fa0] No trailing CRLF found in HTTP header.
            headers = handle_youtubedl_headers(info_dict['http_headers'])
@@ -436,7 +436,15 @@ class FFmpegFD(ExternalFD):
            elif isinstance(conn, compat_str):
                args += ['-rtmp_conn', conn]

-        args += ['-i', url, '-c', 'copy']
+        for url in urls:
+            args += ['-i', url]
+        args += ['-c', 'copy']
+        if info_dict.get('requested_formats'):
+            for (i, fmt) in enumerate(info_dict['requested_formats']):
+                if fmt.get('acodec') != 'none':
+                    args.extend(['-map', '%d:a:0' % i])
+                if fmt.get('vcodec') != 'none':
+                    args.extend(['-map', '%d:v:0' % i])

        if self.params.get('test', False):
            args += ['-fs', compat_str(self._TEST_FILE_SIZE)]
@@ -81,8 +81,6 @@ class HlsFD(FragmentFD):
        man_url = info_dict['url']
        self.to_screen('[%s] Downloading m3u8 manifest' % self.FD_NAME)

-        is_webvtt = info_dict['ext'] == 'vtt'
-
        urlh = self.ydl.urlopen(self._prepare_url(info_dict, man_url))
        man_url = urlh.geturl()
        s = urlh.read().decode('utf-8', 'ignore')
@@ -101,7 +99,11 @@ class HlsFD(FragmentFD):
            #     fd.add_progress_hook(ph)
            return fd.real_download(filename, info_dict)

-        real_downloader = _get_real_downloader(info_dict, 'm3u8_frag_urls', self.params, None)
+        is_webvtt = info_dict['ext'] == 'vtt'
+        if is_webvtt:
+            real_downloader = None  # Packing the fragments is not currently supported for external downloader
+        else:
+            real_downloader = _get_real_downloader(info_dict, 'm3u8_frag_urls', self.params, None)
        if real_downloader and not real_downloader.supports_manifest(s):
            real_downloader = None
        if real_downloader:
@@ -422,8 +424,9 @@ class HlsFD(FragmentFD):
            max_workers = self.params.get('concurrent_fragment_downloads', 1)
            if can_threaded_download and max_workers > 1:
                self.report_warning('The download speed shown is only of one thread. This is a known issue')
+                _download_fragment = lambda f: (f, download_fragment(f)[1])
                with concurrent.futures.ThreadPoolExecutor(max_workers) as pool:
-                    futures = [pool.submit(download_fragment, fragment) for fragment in fragments]
+                    futures = [pool.submit(_download_fragment, fragment) for fragment in fragments]
                    # timeout must be 0 to return instantly
                    done, not_done = concurrent.futures.wait(futures, timeout=0)
                    try:
@@ -437,9 +440,13 @@ class HlsFD(FragmentFD):
                        # timeout must be none to cancel
                        concurrent.futures.wait(not_done, timeout=None)
                        raise KeyboardInterrupt
-                results = [future.result() for future in futures]

-                for frag_content, frag_index in results:
+                for fragment, frag_index in map(lambda x: x.result(), futures):
+                    fragment_filename = '%s-Frag%d' % (ctx['tmpfilename'], frag_index)
+                    down, frag_sanitized = sanitize_open(fragment_filename, 'rb')
+                    fragment['fragment_filename_sanitized'] = frag_sanitized
+                    frag_content = down.read()
+                    down.close()
                    result = append_fragment(frag_content, frag_index)
                    if not result:
                        return False
@@ -1414,7 +1414,7 @@ class AdobePassIE(InfoExtractor):
                authn_token = None
            if not authn_token:
                # TODO add support for other TV Providers
-                mso_id = self._downloader.params.get('ap_mso')
+                mso_id = self.get_param('ap_mso')
                if not mso_id:
                    raise_mvpd_required()
                username, password = self._get_login_info('ap_username', 'ap_password', mso_id)
@@ -323,7 +323,7 @@ class AfreecaTVIE(InfoExtractor):
                        'url': file_url,
                        'format_id': 'http',
                    }]
-                if not formats and not self._downloader.params.get('ignore_no_formats'):
+                if not formats and not self.get_param('ignore_no_formats'):
                    continue
                self._sort_formats(formats)
                file_info = common_entry.copy()
@@ -1,22 +1,36 @@
+# coding: utf-8
 from __future__ import unicode_literals

 import re
 import json

 from .common import InfoExtractor
-from ..compat import compat_urllib_parse_unquote_plus
+from .youtube import YoutubeIE
+from ..compat import (
+    compat_urllib_parse_unquote,
+    compat_urllib_parse_unquote_plus,
+    compat_urlparse,
+    compat_parse_qs,
+    compat_HTTPError
+)
 from ..utils import (
-    KNOWN_EXTENSIONS,
-
+    clean_html,
+    determine_ext,
+    dict_get,
    extract_attributes,
+    ExtractorError,
+    HEADRequest,
+    int_or_none,
+    KNOWN_EXTENSIONS,
+    merge_dicts,
+    mimetype2ext,
+    parse_duration,
+    RegexNotFoundError,
+    str_to_int,
+    str_or_none,
+    try_get,
    unified_strdate,
    unified_timestamp,
-    clean_html,
-    dict_get,
-    parse_duration,
-    int_or_none,
-    str_or_none,
-    merge_dicts,
 )


@@ -241,3 +255,165 @@ class ArchiveOrgIE(InfoExtractor):
                    'parent': 'root'})

        return info
+
+
+class YoutubeWebArchiveIE(InfoExtractor):
+    IE_NAME = 'web.archive:youtube'
+    IE_DESC = 'web.archive.org saved youtube videos'
+    _VALID_URL = r"""(?x)^
+                (?:https?://)?web\.archive\.org/
+                    (?:web/)?
+                    (?:[0-9A-Za-z_*]+/)?  # /web and the version index is optional
+
+                (?:https?(?::|%3[Aa])//)?
+                (?:
+                    (?:\w+\.)?youtube\.com/watch(?:\?|%3[fF])(?:[^\#]+(?:&|%26))?v(?:=|%3[dD])  # Youtube URL
+                    |(wayback-fakeurl\.archive\.org/yt/)  # Or the internal fake url
+                )
+                (?P<id>[0-9A-Za-z_-]{11})(?:%26|\#|&|$)
+                """
+
+    _TESTS = [
+        {
+            'url': 'https://web.archive.org/web/20150415002341/https://www.youtube.com/watch?v=aYAGB11YrSs',
+            'info_dict': {
+                'id': 'aYAGB11YrSs',
+                'ext': 'webm',
+                'title': 'Team Fortress 2 - Sandviches!'
+            }
+        },
+        {
+            # Internal link
+            'url': 'https://web.archive.org/web/2oe/http://wayback-fakeurl.archive.org/yt/97t7Xj_iBv0',
+            'info_dict': {
+                'id': '97t7Xj_iBv0',
+                'ext': 'mp4',
+                'title': 'How Flexible Machines Could Save The World'
+            }
+        },
+        {
+            # Video from 2012, webm format itag 45.
+            'url': 'https://web.archive.org/web/20120712231619/http://www.youtube.com/watch?v=AkhihxRKcrs&gl=US&hl=en',
+            'info_dict': {
+                'id': 'AkhihxRKcrs',
+                'ext': 'webm',
+                'title': 'Limited Run: Mondo\'s Modern Classic 1 of 3 (SDCC 2012)'
+            }
+        },
+        {
+            # Old flash-only video. Webpage title starts with "YouTube - ".
+            'url': 'https://web.archive.org/web/20081211103536/http://www.youtube.com/watch?v=jNQXAC9IVRw',
+            'info_dict': {
+                'id': 'jNQXAC9IVRw',
+                'ext': 'unknown_video',
+                'title': 'Me at the zoo'
+            }
+        },
+        {
+            # Flash video with .flv extension (itag 34). Title has prefix "YouTube         -"
+            # Title has some weird unicode characters too.
+            'url': 'https://web.archive.org/web/20110712231407/http://www.youtube.com/watch?v=lTx3G6h2xyA',
+            'info_dict': {
+                'id': 'lTx3G6h2xyA',
+                'ext': 'flv',
+                'title': '‪Madeon - Pop Culture (live mashup)‬‏'
+            }
+        },
+        {   # Some versions of Youtube have have "YouTube" as page title in html (and later rewritten by js).
+            'url': 'https://web.archive.org/web/http://www.youtube.com/watch?v=kH-G_aIBlFw',
+            'info_dict': {
+                'id': 'kH-G_aIBlFw',
+                'ext': 'mp4',
+                'title': 'kH-G_aIBlFw'
+            },
+            'expected_warnings': [
+                'unable to extract title',
+            ]
+        },
+        {
+            # First capture is a 302 redirect intermediary page.
+            'url': 'https://web.archive.org/web/20050214000000/http://www.youtube.com/watch?v=0altSZ96U4M',
+            'info_dict': {
+                'id': '0altSZ96U4M',
+                'ext': 'mp4',
+                'title': '0altSZ96U4M'
+            },
+            'expected_warnings': [
+                'unable to extract title',
+            ]
+        },
+        {
+            # Video not archived, only capture is unavailable video page
+            'url': 'https://web.archive.org/web/20210530071008/https://www.youtube.com/watch?v=lHJTf93HL1s&spfreload=10',
+            'only_matching': True,
+        },
+        {   # Encoded url
+            'url': 'https://web.archive.org/web/20120712231619/http%3A//www.youtube.com/watch%3Fgl%3DUS%26v%3DAkhihxRKcrs%26hl%3Den',
+            'only_matching': True,
+        },
+        {
+            'url': 'https://web.archive.org/web/20120712231619/http%3A//www.youtube.com/watch%3Fv%3DAkhihxRKcrs%26gl%3DUS%26hl%3Den',
+            'only_matching': True,
+        }
+    ]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        title = video_id  # if we are not able get a title
+
+        def _extract_title(webpage):
+            page_title = self._html_search_regex(
+                r'<title>([^<]*)</title>', webpage, 'title', fatal=False) or ''
+            # YouTube video pages appear to always have either 'YouTube -' as suffix or '- YouTube' as prefix.
+            try:
+                page_title = self._html_search_regex(
+                    r'(?:YouTube\s*-\s*(.*)$)|(?:(.*)\s*-\s*YouTube$)',
+                    page_title, 'title', default='')
+            except RegexNotFoundError:
+                page_title = None
+
+            if not page_title:
+                self.report_warning('unable to extract title', video_id=video_id)
+                return
+            return page_title
+
+        # If the video is no longer available, the oldest capture may be one before it was removed.
+        # Setting the capture date in url to early date seems to redirect to earliest capture.
+        webpage = self._download_webpage(
+            'https://web.archive.org/web/20050214000000/http://www.youtube.com/watch?v=%s' % video_id,
+            video_id=video_id, fatal=False, errnote='unable to download video webpage (probably not archived).')
+        if webpage:
+            title = _extract_title(webpage) or title
+
+        # Use link translator mentioned in https://github.com/ytdl-org/youtube-dl/issues/13655
+        internal_fake_url = 'https://web.archive.org/web/2oe_/http://wayback-fakeurl.archive.org/yt/%s' % video_id
+        try:
+            video_file_webpage = self._request_webpage(
+                HEADRequest(internal_fake_url), video_id,
+                note='Fetching video file url', expected_status=True)
+        except ExtractorError as e:
+            # HTTP Error 404 is expected if the video is not saved.
+            if isinstance(e.cause, compat_HTTPError) and e.cause.code == 404:
+                raise ExtractorError(
+                    'HTTP Error %s. Most likely the video is not archived or issue with web.archive.org.' % e.cause.code,
+                    expected=True)
+            raise
+        video_file_url = compat_urllib_parse_unquote(video_file_webpage.url)
+        video_file_url_qs = compat_parse_qs(compat_urlparse.urlparse(video_file_url).query)
+
+        # Attempt to recover any ext & format info from playback url
+        format = {'url': video_file_url}
+        itag = try_get(video_file_url_qs, lambda x: x['itag'][0])
+        if itag and itag in YoutubeIE._formats:  # Naughty access but it works
+            format.update(YoutubeIE._formats[itag])
+            format.update({'format_id': itag})
+        else:
+            mime = try_get(video_file_url_qs, lambda x: x['mime'][0])
+            ext = mimetype2ext(mime) or determine_ext(video_file_url)
+            format.update({'ext': ext})
+        return {
+            'id': video_id,
+            'title': title,
+            'formats': [format],
+            'duration': str_to_int(try_get(video_file_url_qs, lambda x: x['dur'][0]))
+        }
@@ -290,14 +290,14 @@ class ARDMediathekIE(ARDMediathekBaseIE):


 class ARDIE(InfoExtractor):
-    _VALID_URL = r'(?P<mainurl>https?://(?:www\.)?daserste\.de/[^?#]+/videos(?:extern)?/(?P<display_id>[^/?#]+)-(?:video-?)?(?P<id>[0-9]+))\.html'
+    _VALID_URL = r'(?P<mainurl>https?://(?:www\.)?daserste\.de/(?:[^/?#&]+/)+(?P<id>[^/?#&]+))\.html'
    _TESTS = [{
        # available till 7.01.2022
        'url': 'https://www.daserste.de/information/talk/maischberger/videos/maischberger-die-woche-video100.html',
        'md5': '867d8aa39eeaf6d76407c5ad1bb0d4c1',
        'info_dict': {
-            'display_id': 'maischberger-die-woche',
-            'id': '100',
+            'id': 'maischberger-die-woche-video100',
+            'display_id': 'maischberger-die-woche-video100',
            'ext': 'mp4',
            'duration': 3687.0,
            'title': 'maischberger. die woche vom 7. Januar 2021',
@@ -305,16 +305,28 @@ class ARDIE(InfoExtractor):
            'thumbnail': r're:^https?://.*\.jpg$',
        },
    }, {
-        'url': 'https://www.daserste.de/information/reportage-dokumentation/erlebnis-erde/videosextern/woelfe-und-herdenschutzhunde-ungleiche-brueder-102.html',
+        'url': 'https://www.daserste.de/information/politik-weltgeschehen/morgenmagazin/videosextern/dominik-kahun-aus-der-nhl-direkt-zur-weltmeisterschaft-100.html',
+        'only_matching': True,
+    }, {
+        'url': 'https://www.daserste.de/information/nachrichten-wetter/tagesthemen/videosextern/tagesthemen-17736.html',
+        'only_matching': True,
+    }, {
+        'url': 'https://www.daserste.de/unterhaltung/serie/in-aller-freundschaft-die-jungen-aerzte/videos/diversity-tag-sanam-afrashteh100.html',
        'only_matching': True,
    }, {
        'url': 'http://www.daserste.de/information/reportage-dokumentation/dokus/videos/die-story-im-ersten-mission-unter-falscher-flagge-100.html',
        'only_matching': True,
+    }, {
+        'url': 'https://www.daserste.de/unterhaltung/serie/in-aller-freundschaft-die-jungen-aerzte/Drehpause-100.html',
+        'only_matching': True,
+    }, {
+        'url': 'https://www.daserste.de/unterhaltung/film/filmmittwoch-im-ersten/videos/making-ofwendezeit-video-100.html',
+        'only_matching': True,
    }]

    def _real_extract(self, url):
        mobj = re.match(self._VALID_URL, url)
-        display_id = mobj.group('display_id')
+        display_id = mobj.group('id')

        player_url = mobj.group('mainurl') + '~playerXml.xml'
        doc = self._download_xml(player_url, display_id)
@@ -365,7 +377,7 @@ class ARDIE(InfoExtractor):
        self._sort_formats(formats)

        return {
-            'id': mobj.group('id'),
+            'id': xpath_text(video_node, './videoId', default=display_id),
            'formats': formats,
            'display_id': display_id,
            'title': video_node.find('./title').text,
@@ -245,3 +245,31 @@ class AudiusPlaylistIE(AudiusBaseIE):
        return self.playlist_result(entries, playlist_id,
                                    playlist_data.get('playlist_name', title),
                                    playlist_data.get('description'))
+
+
+class AudiusProfileIE(AudiusPlaylistIE):
+    IE_NAME = 'audius:artist'
+    IE_DESC = 'Audius.co profile/artist pages'
+    _VALID_URL = r'https?://(?:www)?audius\.co/(?P<id>[^\/]+)/?(?:[?#]|$)'
+    _TEST = {
+        'url': 'https://audius.co/pzl/',
+        'info_dict': {
+            'id': 'ezRo7',
+            'description': 'TAMALE\n\nContact: officialpzl@gmail.com',
+            'title': 'pzl',
+        },
+        'playlist_count': 24,
+    }
+
+    def _real_extract(self, url):
+        self._select_api_base()
+        profile_id = self._match_id(url)
+        try:
+            _profile_data = self._api_request('/full/users/handle/' + profile_id, profile_id)
+        except ExtractorError as e:
+            raise ExtractorError('Could not download profile info; ' + str(e))
+        profile_audius_id = _profile_data[0]['id']
+        profile_bio = _profile_data[0].get('bio')
+
+        api_call = self._api_request('/full/users/handle/%s/tracks' % profile_id, profile_id)
+        return self.playlist_result(self._build_playlist(api_call), profile_audius_id, profile_id, profile_bio)
@@ -1271,7 +1271,7 @@ class BBCIE(BBCCoUkIE):
        entries = []
        for num, media_meta in enumerate(medias, start=1):
            formats, subtitles = self._extract_from_media_meta(media_meta, playlist_id)
-            if not formats and not self._downloader.params.get('ignore_no_formats'):
+            if not formats and not self.get_param('ignore_no_formats'):
                continue
            self._sort_formats(formats)

@@ -153,7 +153,7 @@ class BiliBiliIE(InfoExtractor):
        # Bilibili anthologies are similar to playlists but all videos share the same video ID as the anthology itself.
        # If the video has no page argument, check to see if it's an anthology
        if page_id is None:
-            if not self._downloader.params.get('noplaylist'):
+            if not self.get_param('noplaylist'):
                r = self._extract_anthology_entries(bv_id, video_id, webpage)
                if r is not None:
                    self.to_screen('Downloading anthology %s - add --no-playlist to just download video' % video_id)
@@ -299,7 +299,7 @@ class BiliBiliIE(InfoExtractor):
            'tags': tags,
            'raw_tags': raw_tags,
        }
-        if self._downloader.params.get('getcomments', False):
+        if self.get_param('getcomments', False):
            def get_comments():
                comments = self._get_all_comment_pages(video_id)
                return {
@@ -478,7 +478,7 @@ class BrightcoveNewIE(AdobePassIE):
            container = source.get('container')
            ext = mimetype2ext(source.get('type'))
            src = source.get('src')
-            skip_unplayable = not self._downloader.params.get('allow_unplayable_formats')
+            skip_unplayable = not self.get_param('allow_unplayable_formats')
            # https://support.brightcove.com/playback-api-video-fields-reference#key_systems_object
            if skip_unplayable and (container == 'WVM' or source.get('key_systems')):
                num_drm_sources += 1
@@ -547,7 +547,7 @@ class BrightcoveNewIE(AdobePassIE):
                error = errors[0]
                self.raise_no_formats(
                    error.get('message') or error.get('error_subcode') or error['error_code'], expected=True)
-            elif (not self._downloader.params.get('allow_unplayable_formats')
+            elif (not self.get_param('allow_unplayable_formats')
                    and sources and num_drm_sources == len(sources)):
                raise ExtractorError('This video is DRM protected.', expected=True)

@@ -147,7 +147,7 @@ class CeskaTelevizeIE(InfoExtractor):
                is_live = item.get('type') == 'LIVE'
                formats = []
                for format_id, stream_url in item.get('streamUrls', {}).items():
-                    if (not self._downloader.params.get('allow_unplayable_formats')
+                    if (not self.get_param('allow_unplayable_formats')
                            and 'drmOnly=true' in stream_url):
                        continue
                    if 'playerType=flash' in stream_url:
@@ -250,6 +250,8 @@ class InfoExtractor(object):
                    entry and one of:
                        * "data": The subtitles file contents
                        * "url": A URL pointing to the subtitles file
+                    It can optionally also have:
+                        * "name": Name or description of the subtitles
                    "ext" will be calculated from URL if missing
    automatic_captions: Like 'subtitles'; contains automatically generated
                    captions instead of normal subtitles
@@ -420,6 +422,14 @@ class InfoExtractor(object):
    _GEO_IP_BLOCKS = None
    _WORKING = True

+    _LOGIN_HINTS = {
+        'any': 'Use --cookies, --username and --password or --netrc to provide account credentials',
+        'cookies': (
+            'Use --cookies for the authentication. '
+            'See  https://github.com/ytdl-org/youtube-dl#how-do-i-pass-cookies-to-youtube-dl  for how to pass cookies'),
+        'password': 'Use --username and --password or --netrc to provide account credentials',
+    }
+
    def __init__(self, downloader=None):
        """Constructor. Receives an optional downloader."""
        self._ready = False
@@ -489,7 +499,7 @@ class InfoExtractor(object):
        if not self._x_forwarded_for_ip:

            # Geo bypass mechanism is explicitly disabled by user
-            if not self._downloader.params.get('geo_bypass', True):
+            if not self.get_param('geo_bypass', True):
                return

            if not geo_bypass_context:
@@ -511,7 +521,7 @@ class InfoExtractor(object):

            # Explicit IP block specified by user, use it right away
            # regardless of whether extractor is geo bypassable or not
-            ip_block = self._downloader.params.get('geo_bypass_ip_block', None)
+            ip_block = self.get_param('geo_bypass_ip_block', None)

            # Otherwise use random IP block from geo bypass context but only
            # if extractor is known as geo bypassable
@@ -522,17 +532,15 @@ class InfoExtractor(object):

            if ip_block:
                self._x_forwarded_for_ip = GeoUtils.random_ipv4(ip_block)
-                if self._downloader.params.get('verbose', False):
-                    self._downloader.to_screen(
-                        '[debug] Using fake IP %s as X-Forwarded-For.'
-                        % self._x_forwarded_for_ip)
+                self._downloader.write_debug(
+                    '[debug] Using fake IP %s as X-Forwarded-For' % self._x_forwarded_for_ip)
                return

            # Path 2: bypassing based on country code

            # Explicit country code specified by user, use it right away
            # regardless of whether extractor is geo bypassable or not
-            country = self._downloader.params.get('geo_bypass_country', None)
+            country = self.get_param('geo_bypass_country', None)

            # Otherwise use random country code from geo bypass context but
            # only if extractor is known as geo bypassable
@@ -543,10 +551,8 @@ class InfoExtractor(object):

            if country:
                self._x_forwarded_for_ip = GeoUtils.random_ipv4(country)
-                if self._downloader.params.get('verbose', False):
-                    self._downloader.to_screen(
-                        '[debug] Using fake IP %s (%s) as X-Forwarded-For.'
-                        % (self._x_forwarded_for_ip, country.upper()))
+                self._downloader.write_debug(
+                    'Using fake IP %s (%s) as X-Forwarded-For' % (self._x_forwarded_for_ip, country.upper()))

    def extract(self, url):
        """Extracts URL information and returns it in list of dicts."""
@@ -554,12 +560,15 @@ class InfoExtractor(object):
            for _ in range(2):
                try:
                    self.initialize()
+                    self.write_debug('Extracting URL: %s' % url)
                    ie_result = self._real_extract(url)
+                    if ie_result is None:
+                        return None
                    if self._x_forwarded_for_ip:
                        ie_result['__x_forwarded_for_ip'] = self._x_forwarded_for_ip
                    subtitles = ie_result.get('subtitles')
                    if (subtitles and 'live_chat' in subtitles
-                            and 'no-live-chat' in self._downloader.params.get('compat_opts')):
+                            and 'no-live-chat' in self.get_param('compat_opts', [])):
                        del subtitles['live_chat']
                    return ie_result
                except GeoRestrictedError as e:
@@ -574,9 +583,9 @@ class InfoExtractor(object):
            raise ExtractorError('An extractor error has occurred.', cause=e)

    def __maybe_fake_ip_and_retry(self, countries):
-        if (not self._downloader.params.get('geo_bypass_country', None)
+        if (not self.get_param('geo_bypass_country', None)
                and self._GEO_BYPASS
-                and self._downloader.params.get('geo_bypass', True)
+                and self.get_param('geo_bypass', True)
                and not self._x_forwarded_for_ip
                and countries):
            country_code = random.choice(countries)
@@ -630,7 +639,7 @@ class InfoExtractor(object):
        See _download_webpage docstring for arguments specification.
        """
        if not self._downloader._first_webpage_request:
-            sleep_interval = float_or_none(self._downloader.params.get('sleep_interval_requests')) or 0
+            sleep_interval = float_or_none(self.get_param('sleep_interval_requests')) or 0
            if sleep_interval > 0:
                self.to_screen('Sleeping %s seconds ...' % sleep_interval)
                time.sleep(sleep_interval)
@@ -755,11 +764,11 @@ class InfoExtractor(object):
            webpage_bytes = prefix + webpage_bytes
        if not encoding:
            encoding = self._guess_encoding_from_content(content_type, webpage_bytes)
-        if self._downloader.params.get('dump_intermediate_pages', False):
+        if self.get_param('dump_intermediate_pages', False):
            self.to_screen('Dumping request to ' + urlh.geturl())
            dump = base64.b64encode(webpage_bytes).decode('ascii')
            self._downloader.to_screen(dump)
-        if self._downloader.params.get('write_pages', False):
+        if self.get_param('write_pages', False):
            basen = '%s_%s' % (video_id, urlh.geturl())
            if len(basen) > 240:
                h = '___' + hashlib.md5(basen.encode('utf-8')).hexdigest()
@@ -943,14 +952,65 @@ class InfoExtractor(object):
            else:
                self.report_warning(errmsg + str(ve))

-    def report_warning(self, msg, video_id=None):
+    def _parse_socket_response_as_json(self, data, video_id, transform_source=None, fatal=True):
+        return self._parse_json(
+            data[data.find('{'):data.rfind('}') + 1],
+            video_id, transform_source, fatal)
+
+    def _download_socket_json_handle(
+            self, url_or_request, video_id, note='Polling socket',
+            errnote='Unable to poll socket', transform_source=None,
+            fatal=True, encoding=None, data=None, headers={}, query={},
+            expected_status=None):
+        """
+        Return a tuple (JSON object, URL handle).
+
+        See _download_webpage docstring for arguments specification.
+        """
+        res = self._download_webpage_handle(
+            url_or_request, video_id, note, errnote, fatal=fatal,
+            encoding=encoding, data=data, headers=headers, query=query,
+            expected_status=expected_status)
+        if res is False:
+            return res
+        webpage, urlh = res
+        return self._parse_socket_response_as_json(
+            webpage, video_id, transform_source=transform_source,
+            fatal=fatal), urlh
+
+    def _download_socket_json(
+            self, url_or_request, video_id, note='Polling socket',
+            errnote='Unable to poll socket', transform_source=None,
+            fatal=True, encoding=None, data=None, headers={}, query={},
+            expected_status=None):
+        """
+        Return the JSON object as a dict.
+
+        See _download_webpage docstring for arguments specification.
+        """
+        res = self._download_socket_json_handle(
+            url_or_request, video_id, note=note, errnote=errnote,
+            transform_source=transform_source, fatal=fatal, encoding=encoding,
+            data=data, headers=headers, query=query,
+            expected_status=expected_status)
+        return res if res is False else res[0]
+
+    def report_warning(self, msg, video_id=None, *args, **kwargs):
        idstr = '' if video_id is None else '%s: ' % video_id
        self._downloader.report_warning(
-            '[%s] %s%s' % (self.IE_NAME, idstr, msg))
+            '[%s] %s%s' % (self.IE_NAME, idstr, msg), *args, **kwargs)

-    def to_screen(self, msg):
+    def to_screen(self, msg, *args, **kwargs):
        """Print msg to screen, prefixing it with '[ie_name]'"""
-        self._downloader.to_screen('[%s] %s' % (self.IE_NAME, msg))
+        self._downloader.to_screen('[%s] %s' % (self.IE_NAME, msg), *args, **kwargs)
+
+    def write_debug(self, msg, *args, **kwargs):
+        self._downloader.write_debug('[%s] %s' % (self.IE_NAME, msg), *args, **kwargs)
+
+    def get_param(self, name, default=None, *args, **kwargs):
+        if self._downloader:
+            return self._downloader.params.get(name, default, *args, **kwargs)
+        return default

    def report_extraction(self, id_or_name):
        """Report information extraction."""
@@ -969,23 +1029,22 @@ class InfoExtractor(object):
        self.to_screen('Logging in')

    def raise_login_required(
-            self, msg='This video is only available for registered users', metadata_available=False):
-        if metadata_available and self._downloader.params.get('ignore_no_formats_error'):
+            self, msg='This video is only available for registered users',
+            metadata_available=False, method='any'):
+        if metadata_available and self.get_param('ignore_no_formats_error'):
            self.report_warning(msg)
-        raise ExtractorError(
-            '%s. Use --cookies, --username and --password or --netrc to provide account credentials' % msg,
-            expected=True)
+        raise ExtractorError('%s. %s' % (msg, self._LOGIN_HINTS[method]), expected=True)

    def raise_geo_restricted(
            self, msg='This video is not available from your location due to geo restriction',
            countries=None, metadata_available=False):
-        if metadata_available and self._downloader.params.get('ignore_no_formats_error'):
+        if metadata_available and self.get_param('ignore_no_formats_error'):
            self.report_warning(msg)
        else:
            raise GeoRestrictedError(msg, countries=countries)

    def raise_no_formats(self, msg, expected=False, video_id=None):
-        if expected and self._downloader.params.get('ignore_no_formats_error'):
+        if expected and self.get_param('ignore_no_formats_error'):
            self.report_warning(msg, video_id)
        else:
            raise ExtractorError(msg, expected=expected, video_id=video_id)
@@ -1040,7 +1099,7 @@ class InfoExtractor(object):
                if mobj:
                    break

-        if not self._downloader.params.get('no_color') and compat_os_name != 'nt' and sys.stderr.isatty():
+        if not self.get_param('no_color') and compat_os_name != 'nt' and sys.stderr.isatty():
            _name = '\033[0;34m%s\033[0m' % name
        else:
            _name = name
@@ -1074,7 +1133,7 @@ class InfoExtractor(object):
        password = None
        netrc_machine = netrc_machine or self._NETRC_MACHINE

-        if self._downloader.params.get('usenetrc', False):
+        if self.get_param('usenetrc', False):
            try:
                info = netrc.netrc().authenticators(netrc_machine)
                if info is not None:
@@ -1098,15 +1157,11 @@ class InfoExtractor(object):
        value.
        If there's no info available, return (None, None)
        """
-        if self._downloader is None:
-            return (None, None)
-
-        downloader_params = self._downloader.params

        # Attempt to use provided username and password or .netrc data
-        if downloader_params.get(username_option) is not None:
-            username = downloader_params[username_option]
-            password = downloader_params[password_option]
+        username = self.get_param(username_option)
+        if username is not None:
+            password = self.get_param(password_option)
        else:
            username, password = self._get_netrc_login_info(netrc_machine)

@@ -1119,12 +1174,10 @@ class InfoExtractor(object):
        currently just uses the command line option
        If there's no info available, return None
        """
-        if self._downloader is None:
-            return None
-        downloader_params = self._downloader.params

-        if downloader_params.get('twofactor') is not None:
-            return downloader_params['twofactor']
+        tfa = self.get_param('twofactor')
+        if tfa is not None:
+            return tfa

        return compat_getpass('Type %s and press [Return]: ' % note)

@@ -1594,12 +1647,12 @@ class InfoExtractor(object):
                             else limits[0] if has_limit and not has_multiple_limits
                             else None)

-        def print_verbose_info(self, to_screen):
+        def print_verbose_info(self, write_debug):
            if self._sort_user:
-                to_screen('[debug] Sort order given by user: %s' % ', '.join(self._sort_user))
+                write_debug('Sort order given by user: %s' % ', '.join(self._sort_user))
            if self._sort_extractor:
-                to_screen('[debug] Sort order given by extractor: %s' % ', '.join(self._sort_extractor))
-            to_screen('[debug] Formats sorted by: %s' % ', '.join(['%s%s%s' % (
+                write_debug('Sort order given by extractor: %s' % ', '.join(self._sort_extractor))
+            write_debug('Formats sorted by: %s' % ', '.join(['%s%s%s' % (
                '+' if self._get_field_setting(field, 'reverse') else '', field,
                '%s%s(%s)' % ('~' if self._get_field_setting(field, 'closest') else ':',
                              self._get_field_setting(field, 'limit_text'),
@@ -1685,13 +1738,13 @@ class InfoExtractor(object):

    def _sort_formats(self, formats, field_preference=[]):
        if not formats:
-            if self._downloader.params.get('ignore_no_formats_error'):
+            if self.get_param('ignore_no_formats_error'):
                return
            raise ExtractorError('No video formats found')
        format_sort = self.FormatSort()  # params and to_screen are taken from the downloader
        format_sort.evaluate_params(self._downloader.params, field_preference)
-        if self._downloader.params.get('verbose', False):
-            format_sort.print_verbose_info(self._downloader.to_screen)
+        if self.get_param('verbose', False):
+            format_sort.print_verbose_info(self._downloader.write_debug)
        formats.sort(key=lambda f: format_sort.calculate_preference(f))

    def _check_formats(self, formats, video_id):
@@ -1730,7 +1783,7 @@ class InfoExtractor(object):
        """ Either "http:" or "https:", depending on the user's preferences """
        return (
            'http:'
-            if self._downloader.params.get('prefer_insecure', False)
+            if self.get_param('prefer_insecure', False)
            else 'https:')

    def _proto_relative_url(self, url, scheme=None):
@@ -1892,15 +1945,15 @@ class InfoExtractor(object):
        return fmts

    def _extract_m3u8_formats_and_subtitles(
-            self, m3u8_url, video_id, ext=None, entry_protocol='m3u8',
+            self, m3u8_url, video_id, ext=None, entry_protocol='m3u8_native',
            preference=None, quality=None, m3u8_id=None, note=None,
            errnote=None, fatal=True, live=False, data=None, headers={},
            query={}):

        res = self._download_webpage_handle(
            m3u8_url, video_id,
-            note=note or 'Downloading m3u8 information',
-            errnote=errnote or 'Failed to download m3u8 information',
+            note='Downloading m3u8 information' if note is None else note,
+            errnote='Failed to download m3u8 information' if errnote is None else errnote,
            fatal=fatal, data=data, headers=headers, query=query)

        if res is False:
@@ -1916,7 +1969,7 @@ class InfoExtractor(object):
            headers=headers, query=query, video_id=video_id)

    def _parse_m3u8_formats_and_subtitles(
-            self, m3u8_doc, m3u8_url, ext=None, entry_protocol='m3u8',
+            self, m3u8_doc, m3u8_url, ext=None, entry_protocol='m3u8_native',
            preference=None, quality=None, m3u8_id=None, live=False, note=None,
            errnote=None, fatal=True, data=None, headers={}, query={},
            video_id=None):
@@ -1924,7 +1977,7 @@ class InfoExtractor(object):
        if '#EXT-X-FAXS-CM:' in m3u8_doc:  # Adobe Flash Access
            return [], {}

-        if (not self._downloader.params.get('allow_unplayable_formats')
+        if (not self.get_param('allow_unplayable_formats')
                and re.search(r'#EXT-X-SESSION-KEY:.*?URI="skd://', m3u8_doc)):  # Apple FairPlay
            return [], {}

@@ -1937,7 +1990,7 @@ class InfoExtractor(object):
            if re.match(r'^https?://', u)
            else compat_urlparse.urljoin(m3u8_url, u))

-        split_discontinuity = self._downloader.params.get('hls_split_discontinuity', False)
+        split_discontinuity = self.get_param('hls_split_discontinuity', False)

        # References:
        # 1. https://tools.ietf.org/html/draft-pantos-http-live-streaming-21
@@ -2032,7 +2085,12 @@ class InfoExtractor(object):
            groups.setdefault(group_id, []).append(media)
            # <https://tools.ietf.org/html/rfc8216#section-4.3.4.1>
            if media_type == 'SUBTITLES':
-                lang = media['LANGUAGE']  # XXX: normalise?
+                # According to RFC 8216 §4.3.4.2.1, URI is REQUIRED in the
+                # EXT-X-MEDIA tag if the media type is SUBTITLES.
+                # However, lack of URI has been spotted in the wild.
+                # e.g. NebulaIE; see https://github.com/yt-dlp/yt-dlp/issues/339
+                if not media.get('URI'):
+                    return
                url = format_url(media['URI'])
                sub_info = {
                    'url': url,
@@ -2044,6 +2102,7 @@ class InfoExtractor(object):
                    # <https://tools.ietf.org/html/rfc8216#section-3.1>
                    sub_info['ext'] = 'vtt'
                    sub_info['protocol'] = 'm3u8_native'
+                lang = media.get('LANGUAGE') or 'und'
                subtitles.setdefault(lang, []).append(sub_info)
            if media_type not in ('VIDEO', 'AUDIO'):
                return
@@ -2449,8 +2508,8 @@ class InfoExtractor(object):
            fatal=True, data=None, headers={}, query={}):
        res = self._download_xml_handle(
            mpd_url, video_id,
-            note=note or 'Downloading MPD manifest',
-            errnote=errnote or 'Failed to download MPD manifest',
+            note='Downloading MPD manifest' if note is None else note,
+            errnote='Failed to download MPD manifest' if errnote is None else errnote,
            fatal=fatal, data=data, headers=headers, query=query)
        if res is False:
            return [], {}
@@ -2480,7 +2539,7 @@ class InfoExtractor(object):
            http://standards.iso.org/ittf/PubliclyAvailableStandards/c065274_ISO_IEC_23009-1_2014.zip
         2. https://en.wikipedia.org/wiki/Dynamic_Adaptive_Streaming_over_HTTP
        """
-        if not self._downloader.params.get('dynamic_mpd', True):
+        if not self.get_param('dynamic_mpd', True):
            if mpd_doc.get('type') == 'dynamic':
                return [], {}

@@ -2550,7 +2609,7 @@ class InfoExtractor(object):
                        extract_Initialization(segment_template)
            return ms_info

-        skip_unplayable = not self._downloader.params.get('allow_unplayable_formats')
+        skip_unplayable = not self.get_param('allow_unplayable_formats')

        mpd_duration = parse_duration(mpd_doc.get('mediaPresentationDuration'))
        formats = []
@@ -2779,8 +2838,8 @@ class InfoExtractor(object):
    def _extract_ism_formats_and_subtitles(self, ism_url, video_id, ism_id=None, note=None, errnote=None, fatal=True, data=None, headers={}, query={}):
        res = self._download_xml_handle(
            ism_url, video_id,
-            note=note or 'Downloading ISM manifest',
-            errnote=errnote or 'Failed to download ISM manifest',
+            note='Downloading ISM manifest' if note is None else note,
+            errnote='Failed to download ISM manifest' if errnote is None else errnote,
            fatal=fatal, data=data, headers=headers, query=query)
        if res is False:
            return [], {}
@@ -2799,7 +2858,7 @@ class InfoExtractor(object):
        """
        if ism_doc.get('IsLive') == 'TRUE':
            return [], {}
-        if (not self._downloader.params.get('allow_unplayable_formats')
+        if (not self.get_param('allow_unplayable_formats')
                and ism_doc.find('Protection') is not None):
            return [], {}

@@ -3404,8 +3463,8 @@ class InfoExtractor(object):
        return not any_restricted

    def extract_subtitles(self, *args, **kwargs):
-        if (self._downloader.params.get('writesubtitles', False)
-                or self._downloader.params.get('listsubtitles')):
+        if (self.get_param('writesubtitles', False)
+                or self.get_param('listsubtitles')):
            return self._get_subtitles(*args, **kwargs)
        return {}

@@ -3440,8 +3499,8 @@ class InfoExtractor(object):
        return target

    def extract_automatic_captions(self, *args, **kwargs):
-        if (self._downloader.params.get('writeautomaticsub', False)
-                or self._downloader.params.get('listsubtitles')):
+        if (self.get_param('writeautomaticsub', False)
+                or self.get_param('listsubtitles')):
            return self._get_automatic_captions(*args, **kwargs)
        return {}

@@ -3449,9 +3508,9 @@ class InfoExtractor(object):
        raise NotImplementedError('This method must be implemented by subclasses')

    def mark_watched(self, *args, **kwargs):
-        if (self._downloader.params.get('mark_watched', False)
+        if (self.get_param('mark_watched', False)
                and (self._get_login_info()[0] is not None
-                     or self._downloader.params.get('cookiefile') is not None)):
+                     or self.get_param('cookiefile') is not None)):
            self._mark_watched(*args, **kwargs)

    def _mark_watched(self, *args, **kwargs):
@@ -3459,7 +3518,7 @@ class InfoExtractor(object):

    def geo_verification_headers(self):
        headers = {}
-        geo_verification_proxy = self._downloader.params.get('geo_verification_proxy')
+        geo_verification_proxy = self.get_param('geo_verification_proxy')
        if geo_verification_proxy:
            headers['Ytdl-request-proxy'] = geo_verification_proxy
        return headers
@@ -3471,7 +3530,7 @@ class InfoExtractor(object):
        return compat_urllib_parse_unquote(os.path.splitext(url_basename(url))[0])

    @staticmethod
-    def _availability(is_private, needs_premium, needs_subscription, needs_auth, is_unlisted):
+    def _availability(is_private=None, needs_premium=None, needs_subscription=None, needs_auth=None, is_unlisted=None):
        all_known = all(map(
            lambda x: x is not None,
            (is_private, needs_premium, needs_subscription, needs_auth, is_unlisted)))
@@ -26,8 +26,8 @@ class CommonMistakesIE(InfoExtractor):
            'That doesn\'t make any sense. '
            'Simply remove the parameter in your command or configuration.'
        ) % url
-        if not self._downloader.params.get('verbose'):
-            msg += ' Add -v to the command line to see what arguments and configuration yt-dlp got.'
+        if not self.get_param('verbose'):
+            msg += ' Add -v to the command line to see what arguments and configuration yt-dlp has'
        raise ExtractorError(msg, expected=True)


@@ -81,7 +81,7 @@ class CrackleIE(InfoExtractor):
    def _real_extract(self, url):
        video_id = self._match_id(url)

-        geo_bypass_country = self._downloader.params.get('geo_bypass_country', None)
+        geo_bypass_country = self.get_param('geo_bypass_country', None)
        countries = orderedSet((geo_bypass_country, 'US', 'AU', 'CA', 'AS', 'FM', 'GU', 'MP', 'PR', 'PW', 'MH', 'VI', ''))
        num_countries, num = len(countries) - 1, 0

@@ -128,8 +128,8 @@ class CrackleIE(InfoExtractor):
            if isinstance(media.get('MediaURLs'), list):
                break

-        ignore_no_formats = self._downloader.params.get('ignore_no_formats_error')
-        allow_unplayable_formats = self._downloader.params.get('allow_unplayable_formats')
+        ignore_no_formats = self.get_param('ignore_no_formats_error')
+        allow_unplayable_formats = self.get_param('allow_unplayable_formats')

        if not media or (not media.get('MediaURLs') and not ignore_no_formats):
            raise ExtractorError(
@@ -42,7 +42,7 @@ class DailymotionBaseInfoExtractor(InfoExtractor):
    def _real_initialize(self):
        cookies = self._get_dailymotion_cookies()
        ff = self._get_cookie_value(cookies, 'ff')
-        self._FAMILY_FILTER = ff == 'on' if ff else age_restricted(18, self._downloader.params.get('age_limit'))
+        self._FAMILY_FILTER = ff == 'on' if ff else age_restricted(18, self.get_param('age_limit'))
        self._set_dailymotion_cookie('ff', 'on' if self._FAMILY_FILTER else 'off')

    def _call_api(self, object_type, xid, object_fields, note, filter_extra=None):
@@ -207,14 +207,14 @@ class DailymotionIE(DailymotionBaseInfoExtractor):
        video_id, playlist_id = re.match(self._VALID_URL, url).groups()

        if playlist_id:
-            if not self._downloader.params.get('noplaylist'):
+            if not self.get_param('noplaylist'):
                self.to_screen('Downloading playlist %s - add --no-playlist to just download video' % playlist_id)
                return self.url_result(
                    'http://www.dailymotion.com/playlist/' + playlist_id,
                    'DailymotionPlaylist', playlist_id)
            self.to_screen('Downloading just video %s because of --no-playlist' % video_id)

-        password = self._downloader.params.get('videopassword')
+        password = self.get_param('videopassword')
        media = self._call_api(
            'media', video_id, '''... on Video {
      %s
@@ -232,7 +232,7 @@ class DailymotionIE(DailymotionBaseInfoExtractor):
      audienceCount
      isOnAir
    }''' % (self._COMMON_MEDIA_FIELDS, self._COMMON_MEDIA_FIELDS), 'Downloading media JSON metadata',
-            'password: "%s"' % self._downloader.params.get('videopassword') if password else None)
+            'password: "%s"' % self.get_param('videopassword') if password else None)
        xid = media['xid']

        metadata = self._download_json(
@@ -158,7 +158,7 @@ class DaumListIE(InfoExtractor):
        query_dict = compat_parse_qs(compat_urlparse.urlparse(url).query)
        if 'clipid' in query_dict:
            clip_id = query_dict['clipid'][0]
-            if self._downloader.params.get('noplaylist'):
+            if self.get_param('noplaylist'):
                self.to_screen('Downloading just video %s because of --no-playlist' % clip_id)
                return self.url_result(DaumClipIE._URL_TEMPLATE % clip_id, 'DaumClip')
            else:
@@ -13,7 +13,7 @@ from ..utils import (

 class DeezerBaseInfoExtractor(InfoExtractor):
    def get_data(self, url):
-        if not self._downloader.params.get('test'):
+        if not self.get_param('test'):
            self.report_warning('For now, this extractor only supports the 30 second previews. Patches welcome!')

        mobj = re.match(self._VALID_URL, url)
@@ -107,8 +107,7 @@ class EggheadLessonIE(EggheadBaseIE):
            ext = determine_ext(format_url)
            if ext == 'm3u8':
                formats.extend(self._extract_m3u8_formats(
-                    format_url, lesson_id, 'mp4', entry_protocol='m3u8',
-                    m3u8_id='hls', fatal=False))
+                    format_url, lesson_id, 'mp4', m3u8_id='hls', fatal=False))
            elif ext == 'mpd':
                formats.extend(self._extract_mpd_formats(
                    format_url, lesson_id, mpd_id='dash', fatal=False))
@@ -6,7 +6,7 @@ from .common import InfoExtractor
 from ..compat import compat_urllib_parse_urlencode
 from ..utils import (
    ExtractorError,
-    unescapeHTML
+    merge_dicts,
 )


@@ -24,7 +24,8 @@ class EroProfileIE(InfoExtractor):
            'title': 'sexy babe softcore',
            'thumbnail': r're:https?://.*\.jpg',
            'age_limit': 18,
-        }
+        },
+        'skip': 'Video not found',
    }, {
        'url': 'http://www.eroprofile.com/m/videos/view/Try-It-On-Pee_cut_2-wmv-4shared-com-file-sharing-download-movie-file',
        'md5': '1baa9602ede46ce904c431f5418d8916',
@@ -77,19 +78,15 @@ class EroProfileIE(InfoExtractor):
            [r"glbUpdViews\s*\('\d*','(\d+)'", r'p/report/video/(\d+)'],
            webpage, 'video id', default=None)

-        video_url = unescapeHTML(self._search_regex(
-            r'<source src="([^"]+)', webpage, 'video url'))
        title = self._html_search_regex(
-            r'Title:</th><td>([^<]+)</td>', webpage, 'title')
-        thumbnail = self._search_regex(
-            r'onclick="showVideoPlayer\(\)"><img src="([^"]+)',
-            webpage, 'thumbnail', fatal=False)
+            (r'Title:</th><td>([^<]+)</td>', r'<h1[^>]*>(.+?)</h1>'),
+            webpage, 'title')

-        return {
+        info = self._parse_html5_media_entries(url, webpage, video_id)[0]
+
+        return merge_dicts(info, {
            'id': video_id,
            'display_id': display_id,
-            'url': video_url,
            'title': title,
-            'thumbnail': thumbnail,
            'age_limit': 18,
-        }
+        })
@@ -67,7 +67,10 @@ from .appletrailers import (
    AppleTrailersSectionIE,
 )
 from .applepodcasts import ApplePodcastsIE
-from .archiveorg import ArchiveOrgIE
+from .archiveorg import (
+    ArchiveOrgIE,
+    YoutubeWebArchiveIE,
+)
 from .arcpublishing import ArcPublishingIE
 from .arkena import ArkenaIE
 from .ard import (
@@ -94,7 +97,8 @@ from .audiomack import AudiomackIE, AudiomackAlbumIE
 from .audius import (
    AudiusIE,
    AudiusTrackIE,
-    AudiusPlaylistIE
+    AudiusPlaylistIE,
+    AudiusProfileIE,
 )
 from .awaan import (
    AWAANIE,
@@ -395,6 +399,7 @@ from .facebook import (
    FacebookIE,
    FacebookPluginsVideoIE,
 )
+from .fancode import FancodeVodIE
 from .faz import FazIE
 from .fc2 import (
    FC2IE,
@@ -500,6 +505,7 @@ from .hotnewhiphop import HotNewHipHopIE
 from .hotstar import (
    HotStarIE,
    HotStarPlaylistIE,
+    HotStarSeriesIE,
 )
 from .howcast import HowcastIE
 from .howstuffworks import HowStuffWorksIE
@@ -952,6 +958,7 @@ from .palcomp3 import (
 )
 from .pandoratv import PandoraTVIE
 from .parliamentliveuk import ParliamentLiveUKIE
+from .parlview import ParlviewIE
 from .patreon import PatreonIE
 from .pbs import PBSIE
 from .pearvideo import PearVideoIE
@@ -983,6 +990,7 @@ from .platzi import (
 from .playfm import PlayFMIE
 from .playplustv import PlayPlusTVIE
 from .plays import PlaysTVIE
+from .playstuff import PlayStuffIE
 from .playtvak import PlaytvakIE
 from .playvid import PlayvidIE
 from .playwire import PlaywireIE
@@ -1116,6 +1124,7 @@ from .safari import (
    SafariApiIE,
    SafariCourseIE,
 )
+from .saitosan import SaitosanIE
 from .samplefocus import SampleFocusIE
 from .sapo import SapoIE
 from .savefrom import SaveFromIE
@@ -1148,6 +1157,7 @@ from .shared import (
    SharedIE,
    VivoIE,
 )
+from .shemaroome import ShemarooMeIE
 from .showroomlive import ShowRoomLiveIE
 from .simplecast import (
    SimplecastIE,
@@ -1181,7 +1191,10 @@ from .slideslive import SlidesLiveIE
 from .slutload import SlutloadIE
 from .snotr import SnotrIE
 from .sohu import SohuIE
-from .sonyliv import SonyLIVIE
+from .sonyliv import (
+    SonyLIVIE,
+    SonyLIVSeriesIE,
+)
 from .soundcloud import (
    SoundcloudEmbedIE,
    SoundcloudIE,
@@ -1289,6 +1302,7 @@ from .telebruxelles import TeleBruxellesIE
 from .telecinco import TelecincoIE
 from .telegraaf import TelegraafIE
 from .telemb import TeleMBIE
+from .telemundo import TelemundoIE
 from .telequebec import (
    TeleQuebecIE,
    TeleQuebecSquatIE,
@@ -1550,7 +1564,10 @@ from .vodlocker import VodlockerIE
 from .vodpl import VODPlIE
 from .vodplatform import VODPlatformIE
 from .voicerepublic import VoiceRepublicIE
-from .voot import VootIE
+from .voot import (
+    VootIE,
+    VootSeriesIE,
+)
 from .voxmedia import (
    VoxMediaVolumeIE,
    VoxMediaIE,
@@ -0,0 +1,91 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+
+from ..compat import compat_str
+from ..utils import (
+    parse_iso8601,
+    ExtractorError,
+    try_get
+)
+
+
+class FancodeVodIE(InfoExtractor):
+    IE_NAME = 'fancode:vod'
+
+    _VALID_URL = r'https?://(?:www\.)?fancode\.com/video/(?P<id>[0-9]+)\b'
+
+    _TESTS = [{
+        'url': 'https://fancode.com/video/15043/match-preview-pbks-vs-mi',
+        'params': {
+            'skip_download': True,
+            'format': 'bestvideo'
+        },
+        'info_dict': {
+            'id': '6249806281001',
+            'ext': 'mp4',
+            'title': 'Match Preview: PBKS vs MI',
+            'thumbnail': r're:^https?://.*\.jpg$',
+            "timestamp": 1619081590,
+            'view_count': int,
+            'like_count': int,
+            'upload_date': '20210422',
+            'uploader_id': '6008340455001'
+        }
+    }, {
+        'url': 'https://fancode.com/video/15043',
+        'only_matching': True,
+    }]
+
+    def _real_extract(self, url):
+
+        BRIGHTCOVE_URL_TEMPLATE = 'https://players.brightcove.net/%s/default_default/index.html?videoId=%s'
+
+        video_id = self._match_id(url)
+        webpage = self._download_webpage(url, video_id)
+        brightcove_user_id = self._html_search_regex(
+            r'(?:https?://)?players\.brightcove\.net/(\d+)/default_default/index(?:\.min)?\.js',
+            webpage, 'user id')
+
+        data = '''{
+            "query":"query Video($id: Int\\u0021, $filter: SegmentFilter) { media(id: $id, filter: $filter) { id contentId title contentId publishedTime totalViews totalUpvotes provider thumbnail { src } mediaSource {brightcove } duration isPremium isUserEntitled tags duration }}",
+            "variables":{
+                "id":%s,
+                "filter":{
+                    "contentDataType":"DEFAULT"
+                }
+            },
+            "operationName":"Video"
+            }''' % video_id
+
+        metadata_json = self._download_json(
+            'https://www.fancode.com/graphql', video_id, data=data.encode(), note='Downloading metadata',
+            headers={
+                'content-type': 'application/json',
+                'origin': 'https://fancode.com',
+                'referer': url,
+            })
+
+        media = try_get(metadata_json, lambda x: x['data']['media'], dict) or {}
+        brightcove_video_id = try_get(media, lambda x: x['mediaSource']['brightcove'], compat_str)
+
+        if brightcove_video_id is None:
+            raise ExtractorError('Unable to extract brightcove Video ID')
+
+        is_premium = media.get('isPremium')
+        if is_premium:
+            self.report_warning('this video requires a premium account', video_id)
+
+        return {
+            '_type': 'url_transparent',
+            'url': BRIGHTCOVE_URL_TEMPLATE % (brightcove_user_id, brightcove_video_id),
+            'ie_key': 'BrightcoveNew',
+            'id': video_id,
+            'title': media['title'],
+            'like_count': media.get('totalUpvotes'),
+            'view_count': media.get('totalViews'),
+            'tags': media.get('tags'),
+            'release_timestamp': parse_iso8601(media.get('publishedTime')),
+            'availability': self._availability(needs_premium=is_premium),
+        }
@@ -126,6 +126,7 @@ from .viqeo import ViqeoIE
 from .expressen import ExpressenIE
 from .zype import ZypeIE
 from .odnoklassniki import OdnoklassnikiIE
+from .vk import VKIE
 from .kinja import KinjaEmbedIE
 from .gedidigital import GediDigitalIE
 from .rcs import RCSEmbedsIE
@@ -2252,6 +2253,10 @@ class GenericIE(InfoExtractor):
            'playlist_mincount': 52,
        },
        {
+            # Sibnet embed (https://help.sibnet.ru/?sibnet_video_embed)
+            'url': 'https://phpbb3.x-tk.ru/bbcode-video-sibnet-t24.html',
+            'only_matching': True,
+        }, {
            # WimTv embed player
            'url': 'http://www.msmotor.tv/wearefmi-pt-2-2021/',
            'info_dict': {
@@ -2370,7 +2375,7 @@ class GenericIE(InfoExtractor):

        parsed_url = compat_urlparse.urlparse(url)
        if not parsed_url.scheme:
-            default_search = self._downloader.params.get('default_search')
+            default_search = self.get_param('default_search')
            if default_search is None:
                default_search = 'fixup_error'

@@ -2461,8 +2466,8 @@ class GenericIE(InfoExtractor):
            info_dict['subtitles'] = subtitles
            return info_dict

-        if not self._downloader.params.get('test', False) and not is_intentional:
-            force = self._downloader.params.get('force_generic_extractor', False)
+        if not self.get_param('test', False) and not is_intentional:
+            force = self.get_param('force_generic_extractor', False)
            self.report_warning(
                '%s on generic information extractor.' % ('Forcing' if force else 'Falling back'))

@@ -2803,6 +2808,11 @@ class GenericIE(InfoExtractor):
        if odnoklassniki_url:
            return self.url_result(odnoklassniki_url, OdnoklassnikiIE.ie_key())

+        # Look for sibnet embedded player
+        sibnet_urls = VKIE._extract_sibnet_urls(webpage)
+        if sibnet_urls:
+            return self.playlist_from_matches(sibnet_urls, video_id, video_title)
+
        # Look for embedded ivi player
        mobj = re.search(r'<embed[^>]+?src=(["\'])(?P<url>https?://(?:www\.)?ivi\.ru/video/player.+?)\1', webpage)
        if mobj is not None:
@@ -3454,6 +3464,9 @@ class GenericIE(InfoExtractor):
                        'url': src,
                        'ext': (mimetype2ext(src_type)
                                or ext if ext in KNOWN_EXTENSIONS else 'mp4'),
+                        'http_headers': {
+                            'Referer': full_response.geturl(),
+                        },
                    })
            if formats:
                self._sort_formats(formats)
@@ -3522,7 +3535,7 @@ class GenericIE(InfoExtractor):
            m_video_type = re.findall(r'<meta.*?property="og:video:type".*?content="video/(.*?)"', webpage)
            # We only look in og:video if the MIME type is a video, don't try if it's a Flash player:
            if m_video_type is not None:
-                found = filter_video(re.findall(r'<meta.*?property="og:video".*?content="(.*?)"', webpage))
+                found = filter_video(re.findall(r'<meta.*?property="og:(?:video|audio)".*?content="(.*?)"', webpage))
        if not found:
            REDIRECT_REGEX = r'[0-9]{,2};\s*(?:URL|url)=\'?([^\'"]+)'
            found = re.search(
@@ -96,7 +96,7 @@ class GloboIE(InfoExtractor):
        video = self._download_json(
            'http://api.globovideos.com/videos/%s/playlist' % video_id,
            video_id)['videos'][0]
-        if not self._downloader.params.get('allow_unplayable_formats') and video.get('encrypted') is True:
+        if not self.get_param('allow_unplayable_formats') and video.get('encrypted') is True:
            raise ExtractorError('This video is DRM protected.', expected=True)

        title = video['title']
@@ -87,7 +87,14 @@ class HotStarBaseIE(InfoExtractor):

 class HotStarIE(HotStarBaseIE):
    IE_NAME = 'hotstar'
-    _VALID_URL = r'https?://(?:www\.)?hotstar\.com/.*(?P<id>\d{10})'
+    _VALID_URL = r'''(?x)
+                           https?://(?:www\.)?hotstar\.com(?:/in)?/(?!in/)
+                           (?:
+                               tv/(?:[^/?#]+/){3}|
+                               (?!tv/)[^?#]+/
+                           )?
+                           (?P<id>\d{10})
+                   '''
    _TESTS = [{
        # contentData
        'url': 'https://www.hotstar.com/can-you-not-spread-rumours/1000076273',
@@ -141,7 +148,7 @@ class HotStarIE(HotStarBaseIE):

        title = video_data['title']

-        if not self._downloader.params.get('allow_unplayable_formats') and video_data.get('drmProtected'):
+        if not self.get_param('allow_unplayable_formats') and video_data.get('drmProtected'):
            raise ExtractorError('This video is DRM protected.', expected=True)

        headers = {'Referer': url}
@@ -235,3 +242,41 @@ class HotStarPlaylistIE(HotStarBaseIE):
            if video.get('contentId')]

        return self.playlist_result(entries, playlist_id)
+
+
+class HotStarSeriesIE(HotStarBaseIE):
+    IE_NAME = 'hotstar:series'
+    _VALID_URL = r'(?:https?://)(?:www\.)?hotstar\.com(?:/in)?/tv/[^/]+/(?P<id>\d{10})$'
+    _TESTS = [{
+        'url': 'https://www.hotstar.com/in/tv/radhakrishn/1260000646',
+        'info_dict': {
+            'id': '1260000646',
+        },
+        'playlist_mincount': 690,
+    }, {
+        'url': 'https://www.hotstar.com/tv/dancee-/1260050431',
+        'info_dict': {
+            'id': '1260050431',
+        },
+        'playlist_mincount': 43,
+    }]
+
+    def _real_extract(self, url):
+        series_id = self._match_id(url)
+        headers = {
+            'x-country-code': 'IN',
+            'x-platform-code': 'PCTV',
+        }
+        detail_json = self._download_json('https://api.hotstar.com/o/v1/show/detail?contentId=' + series_id,
+                                          video_id=series_id, headers=headers)
+        id = compat_str(try_get(detail_json, lambda x: x['body']['results']['item']['id'], int))
+        item_json = self._download_json('https://api.hotstar.com/o/v1/tray/g/1/items?etid=0&tao=0&tas=10000&eid=' + id,
+                                        video_id=series_id, headers=headers)
+        entries = [
+            self.url_result(
+                'https://www.hotstar.com/%d' % video['contentId'],
+                ie=HotStarIE.ie_key(), video_id=video['contentId'])
+            for video in item_json['body']['results']['items']
+            if video.get('contentId')]
+
+        return self.playlist_result(entries, series_id)
@@ -65,7 +65,7 @@ class ImgGamingBaseIE(InfoExtractor):
        domain, media_type, media_id, playlist_id = re.match(self._VALID_URL, url).groups()

        if playlist_id:
-            if self._downloader.params.get('noplaylist'):
+            if self.get_param('noplaylist'):
                self.to_screen('Downloading just video %s because of --no-playlist' % media_id)
            else:
                self.to_screen('Downloading playlist %s - add --no-playlist to just download video' % playlist_id)
@@ -165,7 +165,7 @@ class IviIE(InfoExtractor):
            content_format = f.get('content_format')
            if not f_url:
                continue
-            if (not self._downloader.params.get('allow_unplayable_formats')
+            if (not self.get_param('allow_unplayable_formats')
                    and ('-MDRM-' in content_format or '-FPS-' in content_format)):
                continue
            formats.append({
@@ -309,7 +309,7 @@ class KalturaIE(InfoExtractor):
            if f.get('fileExt') == 'chun':
                continue
            # DRM-protected video, cannot be decrypted
-            if not self._downloader.params.get('allow_unplayable_formats') and f.get('fileExt') == 'wvm':
+            if not self.get_param('allow_unplayable_formats') and f.get('fileExt') == 'wvm':
                continue
            if not f.get('fileExt'):
                # QT indicates QuickTime; some videos have broken fileExt
@@ -98,7 +98,7 @@ class LimelightBaseIE(InfoExtractor):
            stream_url = stream.get('url')
            if not stream_url or stream_url in urls:
                continue
-            if not self._downloader.params.get('allow_unplayable_formats') and stream.get('drmProtected'):
+            if not self.get_param('allow_unplayable_formats') and stream.get('drmProtected'):
                continue
            urls.append(stream_url)
            ext = determine_ext(stream_url)
@@ -163,7 +163,7 @@ class LimelightBaseIE(InfoExtractor):
            if not media_url or media_url in urls:
                continue
            if (format_id in ('Widevine', 'SmoothStreaming')
-                    and not self._downloader.params.get('allow_unplayable_formats', False)):
+                    and not self.get_param('allow_unplayable_formats', False)):
                continue
            urls.append(media_url)
            ext = determine_ext(media_url)
@@ -71,7 +71,7 @@ class LiTVIE(InfoExtractor):

        video_id = self._match_id(url)

-        noplaylist = self._downloader.params.get('noplaylist')
+        noplaylist = self.get_param('noplaylist')
        noplaylist_prompt = True
        if 'force_noplaylist' in data:
            noplaylist = data['force_noplaylist']
@@ -167,7 +167,7 @@ class NBAWatchIE(NBAWatchBaseIE):
        display_id = self._match_id(url)
        collection_id = compat_parse_qs(compat_urllib_parse_urlparse(url).query).get('collection', [None])[0]
        if collection_id:
-            if self._downloader.params.get('noplaylist'):
+            if self.get_param('noplaylist'):
                self.to_screen('Downloading just video %s because of --no-playlist' % display_id)
            else:
                self.to_screen('Downloading playlist %s - add --no-playlist to just download video' % collection_id)
@@ -405,7 +405,7 @@ class NetEaseMusicProgramIE(NetEaseMusicBaseIE):
        name = info['name']
        description = info['description']

-        if not info['songs'] or self._downloader.params.get('noplaylist'):
+        if not info['songs'] or self.get_param('noplaylist'):
            if info['songs']:
                self.to_screen(
                    'Downloading just the main audio %s because of --no-playlist'
@@ -34,7 +34,7 @@ class NineCNineMediaIE(InfoExtractor):
                '$include': '[HasClosedCaptions]',
            })

-        if (not self._downloader.params.get('allow_unplayable_formats')
+        if (not self.get_param('allow_unplayable_formats')
                and try_get(content_package, lambda x: x['Constraints']['Security']['Type'])):
            raise ExtractorError('This video is DRM protected.', expected=True)

@@ -66,7 +66,7 @@ class NineNowIE(InfoExtractor):

        video_data = common_data['video']

-        if not self._downloader.params.get('allow_unplayable_formats') and video_data.get('drm'):
+        if not self.get_param('allow_unplayable_formats') and video_data.get('drm'):
            raise ExtractorError('This video is DRM protected.', expected=True)

        brightcove_id = video_data.get('brightcoveId') or 'ref:' + video_data['referenceId']
@@ -246,7 +246,7 @@ class NPOIE(NPOBaseIE):
                })

        if not formats:
-            if not self._downloader.params.get('allow_unplayable_formats') and drm:
+            if not self.get_param('allow_unplayable_formats') and drm:
                self.raise_no_formats('This video is DRM protected.', expected=True)
            return

@@ -247,8 +247,7 @@ class OdnoklassnikiIE(InfoExtractor):
        m3u8_url = metadata.get('hlsMasterPlaylistUrl')
        if m3u8_url:
            formats.extend(self._extract_m3u8_formats(
-                m3u8_url, video_id, 'mp4', entry_protocol='m3u8',
-                m3u8_id='hls', fatal=False))
+                m3u8_url, video_id, 'mp4', m3u8_id='hls', fatal=False))
        rtmp_url = metadata.get('rtmpUrl')
        if rtmp_url:
            formats.append({
@@ -182,7 +182,7 @@ class OnetChannelIE(OnetBaseIE):
        video_id = remove_start(current_clip_info['ckmId'], 'mvp:')
        video_name = url_basename(current_clip_info['url'])

-        if self._downloader.params.get('noplaylist'):
+        if self.get_param('noplaylist'):
            self.to_screen(
                'Downloading just video %s because of --no-playlist' % video_name)
            return self._extract_from_id(video_id, webpage)
@@ -182,7 +182,7 @@ class ORFRadioIE(InfoExtractor):
            duration = end - start if end and start else None
            entries.append({
                'id': loop_stream_id.replace('.mp3', ''),
-                'url': 'http://loopstream01.apa.at/?channel=%s&id=%s' % (self._LOOP_STATION, loop_stream_id),
+                'url': 'https://loopstream01.apa.at/?channel=%s&id=%s' % (self._LOOP_STATION, loop_stream_id),
                'title': title,
                'description': clean_html(data.get('subtitle')),
                'duration': duration,
@@ -0,0 +1,68 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+from ..compat import compat_str
+from ..utils import (
+    int_or_none,
+    try_get,
+    unified_timestamp,
+)
+
+
+class ParlviewIE(InfoExtractor):
+
+    _VALID_URL = r'https?://(?:www\.)?parlview\.aph\.gov\.au/(?:[^/]+)?\bvideoID=(?P<id>\d{6})'
+    _TESTS = [{
+        'url': 'https://parlview.aph.gov.au/mediaPlayer.php?videoID=542661',
+        'info_dict': {
+            'id': '542661',
+            'ext': 'mp4',
+            'title': "Australia's Family Law System [Part 2]",
+            'duration': 5799,
+            'description': 'md5:7099883b391619dbae435891ca871a62',
+            'timestamp': 1621430700,
+            'upload_date': '20210519',
+            'uploader': 'Joint Committee',
+        },
+        'params': {
+            'skip_download': True,
+        }
+    }, {
+        'url': 'https://parlview.aph.gov.au/mediaPlayer.php?videoID=539936',
+        'only_matching': True,
+    }]
+    _API_URL = 'https://parlview.aph.gov.au/api_v3/1/playback/getUniversalPlayerConfig?videoID=%s&format=json'
+    _MEDIA_INFO_URL = 'https://parlview.aph.gov.au/ajaxPlayer.php?videoID=%s&tabNum=4&action=loadTab'
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        webpage = self._download_webpage(url, video_id)
+        media = self._download_json(self._API_URL % video_id, video_id).get('media')
+        timestamp = try_get(media, lambda x: x['timeMap']['source']['timecode_offsets'][0], compat_str) or '/'
+
+        stream = try_get(media, lambda x: x['renditions'][0], dict)
+        if not stream:
+            self.raise_no_formats('No streams were detected')
+        elif stream.get('streamType') != 'VOD':
+            self.raise_no_formats('Unknown type of stream was detected: "%s"' % str(stream.get('streamType')))
+        formats = self._extract_m3u8_formats(stream['url'], video_id, 'mp4', 'm3u8_native')
+        self._sort_formats(formats)
+
+        media_info = self._download_webpage(
+            self._MEDIA_INFO_URL % video_id, video_id, note='Downloading media info', fatal=False)
+
+        return {
+            'id': video_id,
+            'url': url,
+            'title': self._html_search_regex(r'<h2>([^<]+)<', webpage, 'title', fatal=False),
+            'formats': formats,
+            'duration': int_or_none(media.get('duration')),
+            'timestamp': unified_timestamp(timestamp.split('/', 1)[1].replace('_', ' ')),
+            'description': self._html_search_regex(
+                r'<div[^>]+class="descripti?on"[^>]*>[^>]+<strong>[^>]+>[^>]+>([^<]+)',
+                webpage, 'description', fatal=False),
+            'uploader': self._html_search_regex(
+                r'<td>[^>]+>Channel:[^>]+>([^<]+)', media_info, 'channel', fatal=False),
+            'thumbnail': media.get('staticImage'),
+        }
@@ -2,6 +2,9 @@
 from __future__ import unicode_literals

 from .common import InfoExtractor
+from .vimeo import VimeoIE
+
+from ..compat import compat_urllib_parse_unquote
 from ..utils import (
    clean_html,
    determine_ext,
@@ -11,6 +14,7 @@ from ..utils import (
    parse_iso8601,
    str_or_none,
    try_get,
+    url_or_none
 )


@@ -63,6 +67,20 @@ class PatreonIE(InfoExtractor):
    }, {
        'url': 'https://www.patreon.com/posts/743933',
        'only_matching': True,
+    }, {
+        'url': 'https://www.patreon.com/posts/kitchen-as-seen-51706779',
+        'md5': '96656690071f6d64895866008484251b',
+        'info_dict': {
+            'id': '555089736',
+            'ext': 'mp4',
+            'title': 'KITCHEN AS SEEN ON DEEZ NUTS EXTENDED!',
+            'uploader': 'Cold Ones',
+            'thumbnail': 're:^https?://.*$',
+            'upload_date': '20210526',
+            'description': 'md5:557a409bd79d3898689419094934ba79',
+            'uploader_id': '14936315',
+        },
+        'skip': 'Patron-only content'
    }]

    # Currently Patreon exposes download URL via hidden CSS, so login is not
@@ -136,6 +154,19 @@ class PatreonIE(InfoExtractor):
                        'uploader_url': user_attributes.get('url'),
                    })

+        if not info.get('url'):
+            # handle Vimeo embeds
+            if try_get(attributes, lambda x: x['embed']['provider']) == 'Vimeo':
+                embed_html = try_get(attributes, lambda x: x['embed']['html'])
+                v_url = url_or_none(compat_urllib_parse_unquote(
+                    self._search_regex(r'src=(https%3A%2F%2Fplayer\.vimeo\.com.+)%3F', embed_html, 'vimeo url', fatal=False)))
+                if v_url:
+                    info.update({
+                        '_type': 'url_transparent',
+                        'url': VimeoIE._smuggle_referrer(v_url, 'https://patreon.com'),
+                        'ie_key': 'Vimeo',
+                    })
+
        if not info.get('url'):
            embed_url = try_get(attributes, lambda x: x['embed']['url'])
            if embed_url:
@@ -79,7 +79,7 @@ class PhilharmonieDeParisIE(InfoExtractor):
                formats.extend(self._extract_m3u8_formats(
                    m3u8_url, video_id, 'mp4', entry_protocol='m3u8_native',
                    m3u8_id='hls', fatal=False))
-            if not formats and not self._downloader.params.get('ignore_no_formats'):
+            if not formats and not self.get_param('ignore_no_formats'):
                return
            self._sort_formats(formats)
            return {
@@ -9,8 +9,9 @@ from ..compat import compat_str
 from ..utils import (
    int_or_none,
    merge_dicts,
+    try_get,
    unified_timestamp,
-    xpath_text,
+    urljoin,
 )


@@ -27,10 +28,11 @@ class PhoenixIE(ZDFBaseIE):
            'title': 'Wohin führt der Protest in der Pandemie?',
            'description': 'md5:7d643fe7f565e53a24aac036b2122fbd',
            'duration': 1691,
-            'timestamp': 1613906100,
+            'timestamp': 1613902500,
            'upload_date': '20210221',
            'uploader': 'Phoenix',
-            'channel': 'corona nachgehakt',
+            'series': 'corona nachgehakt',
+            'episode': 'Wohin führt der Protest in der Pandemie?',
        },
    }, {
        # Youtube embed
@@ -79,50 +81,53 @@ class PhoenixIE(ZDFBaseIE):

        video_id = compat_str(video.get('basename') or video.get('content'))

-        details = self._download_xml(
+        details = self._download_json(
            'https://www.phoenix.de/php/mediaplayer/data/beitrags_details.php',
-            video_id, 'Downloading details XML', query={
+            video_id, 'Downloading details JSON', query={
                'ak': 'web',
                'ptmd': 'true',
                'id': video_id,
                'profile': 'player2',
            })

-        title = title or xpath_text(
-            details, './/information/title', 'title', fatal=True)
-        content_id = xpath_text(
-            details, './/video/details/basename', 'content id', fatal=True)
+        title = title or details['title']
+        content_id = details['tracking']['nielsen']['content']['assetid']

        info = self._extract_ptmd(
            'https://tmd.phoenix.de/tmd/2/ngplayer_2_3/vod/ptmd/phoenix/%s' % content_id,
            content_id, None, url)

-        timestamp = unified_timestamp(xpath_text(details, './/details/airtime'))
+        duration = int_or_none(try_get(
+            details, lambda x: x['tracking']['nielsen']['content']['length']))
+        timestamp = unified_timestamp(details.get('editorialDate'))
+        series = try_get(
+            details, lambda x: x['tracking']['nielsen']['content']['program'],
+            compat_str)
+        episode = title if details.get('contentType') == 'episode' else None

        thumbnails = []
-        for node in details.findall('.//teaserimages/teaserimage'):
-            thumbnail_url = node.text
+        teaser_images = try_get(details, lambda x: x['teaserImageRef']['layouts'], dict) or {}
+        for thumbnail_key, thumbnail_url in teaser_images.items():
+            thumbnail_url = urljoin(url, thumbnail_url)
            if not thumbnail_url:
                continue
            thumbnail = {
                'url': thumbnail_url,
            }
-            thumbnail_key = node.get('key')
-            if thumbnail_key:
-                m = re.match('^([0-9]+)x([0-9]+)$', thumbnail_key)
-                if m:
-                    thumbnail['width'] = int(m.group(1))
-                    thumbnail['height'] = int(m.group(2))
+            m = re.match('^([0-9]+)x([0-9]+)$', thumbnail_key)
+            if m:
+                thumbnail['width'] = int(m.group(1))
+                thumbnail['height'] = int(m.group(2))
            thumbnails.append(thumbnail)

        return merge_dicts(info, {
            'id': content_id,
            'title': title,
-            'description': xpath_text(details, './/information/detail'),
-            'duration': int_or_none(xpath_text(details, './/details/lengthSec')),
+            'description': details.get('leadParagraph'),
+            'duration': duration,
            'thumbnails': thumbnails,
            'timestamp': timestamp,
-            'uploader': xpath_text(details, './/details/channel'),
-            'uploader_id': xpath_text(details, './/details/originChannelId'),
-            'channel': xpath_text(details, './/details/originChannelTitle'),
+            'uploader': details.get('tvService'),
+            'series': series,
+            'episode': episode,
        })
@@ -0,0 +1,65 @@
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+from ..compat import compat_str
+from ..utils import (
+    smuggle_url,
+    try_get,
+)
+
+
+class PlayStuffIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www\.)?play\.stuff\.co\.nz/details/(?P<id>[^/?#&]+)'
+    _TESTS = [{
+        'url': 'https://play.stuff.co.nz/details/608778ac1de1c4001a3fa09a',
+        'md5': 'c82d3669e5247c64bc382577843e5bd0',
+        'info_dict': {
+            'id': '6250584958001',
+            'ext': 'mp4',
+            'title': 'Episode 1: Rotorua/Mt Maunganui/Tauranga',
+            'description': 'md5:c154bafb9f0dd02d01fd4100fb1c1913',
+            'uploader_id': '6005208634001',
+            'timestamp': 1619491027,
+            'upload_date': '20210427',
+        },
+        'add_ie': ['BrightcoveNew'],
+    }, {
+        # geo restricted, bypassable
+        'url': 'https://play.stuff.co.nz/details/_6155660351001',
+        'only_matching': True,
+    }]
+    BRIGHTCOVE_URL_TEMPLATE = 'http://players.brightcove.net/%s/%s_default/index.html?videoId=%s'
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+
+        webpage = self._download_webpage(url, video_id)
+
+        state = self._parse_json(
+            self._search_regex(
+                r'__INITIAL_STATE__\s*=\s*({.+?})\s*;', webpage, 'state'),
+            video_id)
+
+        account_id = try_get(
+            state, lambda x: x['configurations']['accountId'],
+            compat_str) or '6005208634001'
+        player_id = try_get(
+            state, lambda x: x['configurations']['playerId'],
+            compat_str) or 'default'
+
+        entries = []
+        for item_id, video in state['items'].items():
+            if not isinstance(video, dict):
+                continue
+            asset_id = try_get(
+                video, lambda x: x['content']['attributes']['assetId'],
+                compat_str)
+            if not asset_id:
+                continue
+            entries.append(self.url_result(
+                smuggle_url(
+                    self.BRIGHTCOVE_URL_TEMPLATE % (account_id, player_id, asset_id),
+                    {'geo_countries': ['NZ']}),
+                'BrightcoveNew', video_id))
+
+        return self.playlist_result(entries, video_id)
@@ -337,11 +337,11 @@ query viewClip {
        # In order to minimize the number of calls to ViewClip API and reduce
        # the probability of being throttled or banned by Pluralsight we will request
        # only single format until formats listing was explicitly requested.
-        if self._downloader.params.get('listformats', False):
+        if self.get_param('listformats', False):
            allowed_qualities = ALLOWED_QUALITIES
        else:
            def guess_allowed_qualities():
-                req_format = self._downloader.params.get('format') or 'best'
+                req_format = self.get_param('format') or 'best'
                req_format_split = req_format.split('-', 1)
                if len(req_format_split) > 1:
                    req_ext, req_quality = req_format_split
@@ -349,7 +349,7 @@ query viewClip {
                    for allowed_quality in ALLOWED_QUALITIES:
                        if req_ext == allowed_quality.ext and req_quality in allowed_quality.qualities:
                            return (AllowedQuality(req_ext, (req_quality, )), )
-                req_ext = 'webm' if self._downloader.params.get('prefer_free_formats') else 'mp4'
+                req_ext = 'webm' if self.get_param('prefer_free_formats') else 'mp4'
                return (AllowedQuality(req_ext, (best_quality, )), )
            allowed_qualities = guess_allowed_qualities()

@@ -109,7 +109,7 @@ class PlutoTVIE(InfoExtractor):
        if ad_free_formats:
            formats, subtitles = ad_free_formats, ad_free_subtitles
        else:
-            self._downloader.report_warning('Unable to find ad-free formats')
+            self.report_warning('Unable to find ad-free formats')
        return formats, subtitles

    def _get_video_info(self, video_json, slug, series_name=None):
@@ -112,7 +112,7 @@ class PokemonWatchIE(InfoExtractor):
        }

        # API call can be avoided entirely if we are listing formats
-        if self._downloader.params.get('listformats', False):
+        if self.get_param('listformats', False):
            return info

        webpage = self._download_webpage(url, video_id)
@@ -34,7 +34,7 @@ class ProSiebenSat1BaseIE(InfoExtractor):
                'ids': clip_id,
            })[0]

-        if not self._downloader.params.get('allow_unplayable_formats') and video.get('is_protected') is True:
+        if not self.get_param('allow_unplayable_formats') and video.get('is_protected') is True:
            raise ExtractorError('This video is DRM protected.', expected=True)

        formats = []
@@ -275,7 +275,7 @@ class RaiPlayIE(RaiBaseIE):
        media = self._download_json(
            base + '.json', video_id, 'Downloading video JSON')

-        if not self._downloader.params.get('allow_unplayable_formats'):
+        if not self.get_param('allow_unplayable_formats'):
            if try_get(
                    media,
                    (lambda x: x['rights_management']['rights']['drm'],
@@ -133,8 +133,10 @@ class RedBullEmbedIE(RedBullTVIE):
        rrn_id = self._match_id(url)
        asset_id = self._download_json(
            'https://edge-graphql.crepo-production.redbullaws.com/v1/graphql',
-            rrn_id, headers={'API-KEY': 'e90a1ff11335423998b100c929ecc866'},
-            query={
+            rrn_id, headers={
+                'Accept': 'application/json',
+                'API-KEY': 'e90a1ff11335423998b100c929ecc866',
+            }, query={
                'query': '''{
  resource(id: "%s", enforceGeoBlocking: false) {
    %s
@@ -13,7 +13,7 @@ from ..utils import smuggle_url


 class RMCDecouverteIE(InfoExtractor):
-    _VALID_URL = r'https?://rmcdecouverte\.bfmtv\.com/(?:[^/]+/(?P<id>[^?#/]+)|(?P<live_id>mediaplayer-direct))'
+    _VALID_URL = r'https?://rmcdecouverte\.bfmtv\.com/(?:[^?#]*_(?P<id>\d+)|mediaplayer-direct)/?(?:[#?]|$)'

    _TESTS = [{
        'url': 'https://rmcdecouverte.bfmtv.com/vestiges-de-guerre_22240/les-bunkers-secrets-domaha-beach_25303/',
@@ -45,6 +45,13 @@ class RMCDecouverteIE(InfoExtractor):
            'skip_download': True,
        },
        'skip': 'only available for a week',
+    }, {
+        'url': 'https://rmcdecouverte.bfmtv.com/avions-furtifs-la-technologie-de-lextreme_10598',
+        'only_matching': True,
+    }, {
+        # The website accepts any URL as long as it has _\d+ at the end
+        'url': 'https://rmcdecouverte.bfmtv.com/any/thing/can/go/here/_10598',
+        'only_matching': True,
    }, {
        # live, geo restricted, bypassable
        'url': 'https://rmcdecouverte.bfmtv.com/mediaplayer-direct/',
@@ -54,7 +61,7 @@ class RMCDecouverteIE(InfoExtractor):

    def _real_extract(self, url):
        mobj = re.match(self._VALID_URL, url)
-        display_id = mobj.group('id') or mobj.group('live_id')
+        display_id = mobj.group('id') or 'direct'
        webpage = self._download_webpage(url, display_id)
        brightcove_legacy_url = BrightcoveLegacyIE._extract_brightcove_url(webpage)
        if brightcove_legacy_url:
@@ -125,7 +125,7 @@ class RTBFIE(InfoExtractor):
                })

        mpd_url = data.get('urlDash')
-        if mpd_url and (self._downloader.params.get('allow_unplayable_formats') or not data.get('drm')):
+        if mpd_url and (self.get_param('allow_unplayable_formats') or not data.get('drm')):
            formats.extend(self._extract_mpd_formats(
                mpd_url, media_id, mpd_id='dash', fatal=False))

@@ -200,7 +200,7 @@ class RuutuIE(InfoExtractor):
                return node.get('value')

        if not formats:
-            if (not self._downloader.params.get('allow_unplayable_formats')
+            if (not self.get_param('allow_unplayable_formats')
                    and xpath_text(video_xml, './Clip/DRM', default=None)):
                self.raise_no_formats('This video is DRM protected.', expected=True)
            ns_st_cds = pv('ns_st_cds')
@@ -0,0 +1,78 @@
+# coding: utf-8
+
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+from ..utils import ExtractorError, try_get
+
+
+class SaitosanIE(InfoExtractor):
+    IE_NAME = 'Saitosan'
+    _VALID_URL = r'https?://(?:www\.)?saitosan\.net/bview.html\?id=(?P<id>[0-9]+)'
+    _TESTS = [{
+        'url': 'http://www.saitosan.net/bview.html?id=10031846',
+        'info_dict': {
+            'id': '10031846',
+            'ext': 'mp4',
+            'title': '井下原 和弥',
+            'uploader': '井下原 和弥',
+            'thumbnail': 'http://111.171.196.85:8088/921f916f-7f55-4c97-b92e-5d9d0fef8f5f/thumb',
+            'is_live': True,
+        },
+        'params': {
+            # m3u8 download
+            'skip_download': True,
+        },
+        'skip': 'Broadcasts are ephemeral',
+    },
+        {
+        'url': 'http://www.saitosan.net/bview.html?id=10031795',
+        'info_dict': {
+            'id': '10031795',
+            'ext': 'mp4',
+            'title': '橋本',
+            'uploader': '橋本',
+            'thumbnail': 'http://111.171.196.85:8088/1a3933e1-a01a-483b-8931-af15f37f8082/thumb',
+            'is_live': True,
+        },
+        'params': {
+            # m3u8 download
+            'skip_download': True,
+        },
+        'skip': 'Broadcasts are ephemeral',
+    }]
+
+    def _real_extract(self, url):
+        b_id = self._match_id(url)
+
+        base = 'http://hankachi.saitosan-api.net:8002/socket.io/?transport=polling&EIO=3'
+        sid = self._download_socket_json(base, b_id, note='Opening socket').get('sid')
+        base += '&sid=' + sid
+
+        self._download_webpage(base, b_id, note='Polling socket')
+        payload = '420["room_start_join",{"room_id":"%s"}]' % b_id
+        payload = '%s:%s' % (len(payload), payload)
+
+        self._download_webpage(base, b_id, data=payload, note='Polling socket with payload')
+        response = self._download_socket_json(base, b_id, note='Polling socket')
+        if not response.get('ok'):
+            err = response.get('error') or {}
+            raise ExtractorError(
+                '%s said: %s - %s' % (self.IE_NAME, err.get('code', '?'), err.get('msg', 'Unknown')) if err
+                else 'The socket reported that the broadcast could not be joined. Maybe it\'s offline or the URL is incorrect',
+                expected=True, video_id=b_id)
+
+        self._download_webpage(base, b_id, data='26:421["room_finish_join",{}]', note='Polling socket')
+        b_data = self._download_socket_json(base, b_id, note='Getting broadcast metadata from socket')
+        m3u8_url = b_data.get('url')
+
+        self._download_webpage(base, b_id, data='1:1', note='Closing socket', fatal=False)
+
+        return {
+            'id': b_id,
+            'title': b_data.get('name'),
+            'formats': self._extract_m3u8_formats(m3u8_url, b_id, 'mp4', live=True),
+            'thumbnail': m3u8_url.replace('av.m3u8', 'thumb'),
+            'uploader': try_get(b_data, lambda x: x['broadcast_user']['name']),  # same as title
+            'is_live': True
+        }
@@ -21,6 +21,7 @@ from ..utils import (
 class ShahidBaseIE(AWSIE):
    _AWS_PROXY_HOST = 'api2.shahid.net'
    _AWS_API_KEY = '2RRtuMHx95aNI1Kvtn2rChEuwsCogUd4samGPjLh'
+    _VALID_URL_BASE = r'https?://shahid\.mbc\.net/[a-z]{2}/'

    def _handle_error(self, e):
        fail_data = self._parse_json(
@@ -49,7 +50,7 @@ class ShahidBaseIE(AWSIE):

 class ShahidIE(ShahidBaseIE):
    _NETRC_MACHINE = 'shahid'
-    _VALID_URL = r'https?://shahid\.mbc\.net/ar/(?:serie|show|movie)s/[^/]+/(?P<type>episode|clip|movie)-(?P<id>\d+)'
+    _VALID_URL = ShahidBaseIE._VALID_URL_BASE + r'(?:serie|show|movie)s/[^/]+/(?P<type>episode|clip|movie)-(?P<id>\d+)'
    _TESTS = [{
        'url': 'https://shahid.mbc.net/ar/shows/%D9%85%D8%AA%D8%AD%D9%81-%D8%A7%D9%84%D8%AF%D8%AD%D9%8A%D8%AD-%D8%A7%D9%84%D9%85%D9%88%D8%B3%D9%85-1-%D9%83%D9%84%D9%8A%D8%A8-1/clip-816924',
        'info_dict': {
@@ -73,6 +74,9 @@ class ShahidIE(ShahidBaseIE):
        # shahid plus subscriber only
        'url': 'https://shahid.mbc.net/ar/series/%D9%85%D8%B1%D8%A7%D9%8A%D8%A7-2011-%D8%A7%D9%84%D9%85%D9%88%D8%B3%D9%85-1-%D8%A7%D9%84%D8%AD%D9%84%D9%82%D8%A9-1/episode-90511',
        'only_matching': True
+    }, {
+        'url': 'https://shahid.mbc.net/en/shows/Ramez-Fi-Al-Shallal-season-1-episode-1/episode-359319',
+        'only_matching': True
    }]

    def _real_initialize(self):
@@ -114,7 +118,7 @@ class ShahidIE(ShahidBaseIE):
        playout = self._call_api(
            'playout/new/url/' + video_id, video_id)['playout']

-        if not self._downloader.params.get('allow_unplayable_formats') and playout.get('drm'):
+        if not self.get_param('allow_unplayable_formats') and playout.get('drm'):
            raise ExtractorError('This video is DRM protected.', expected=True)

        formats = self._extract_m3u8_formats(re.sub(
@@ -168,7 +172,7 @@ class ShahidIE(ShahidBaseIE):


 class ShahidShowIE(ShahidBaseIE):
-    _VALID_URL = r'https?://shahid\.mbc\.net/ar/(?:show|serie)s/[^/]+/(?:show|series)-(?P<id>\d+)'
+    _VALID_URL = ShahidBaseIE._VALID_URL_BASE + r'(?:show|serie)s/[^/]+/(?:show|series)-(?P<id>\d+)'
    _TESTS = [{
        'url': 'https://shahid.mbc.net/ar/shows/%D8%B1%D8%A7%D9%85%D8%B2-%D9%82%D8%B1%D8%B4-%D8%A7%D9%84%D8%A8%D8%AD%D8%B1/show-79187',
        'info_dict': {
@@ -86,10 +86,10 @@ class SharedIE(SharedBaseIE):

 class VivoIE(SharedBaseIE):
    IE_DESC = 'vivo.sx'
-    _VALID_URL = r'https?://vivo\.sx/(?P<id>[\da-z]{10})'
+    _VALID_URL = r'https?://vivo\.s[xt]/(?P<id>[\da-z]{10})'
    _FILE_NOT_FOUND = '>The file you have requested does not exists or has been removed'

-    _TEST = {
+    _TESTS = [{
        'url': 'http://vivo.sx/d7ddda0e78',
        'md5': '15b3af41be0b4fe01f4df075c2678b2c',
        'info_dict': {
@@ -98,7 +98,10 @@ class VivoIE(SharedBaseIE):
            'title': 'Chicken',
            'filesize': 515659,
        },
-    }
+    }, {
+        'url': 'http://vivo.st/d7ddda0e78',
+        'only_matching': True,
+    }]

    def _extract_title(self, webpage):
        title = self._html_search_regex(
@@ -0,0 +1,76 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+import re
+
+from .common import InfoExtractor
+from ..aes import aes_cbc_decrypt
+from ..compat import (
+    compat_b64decode,
+    compat_ord,
+)
+from ..utils import (
+    bytes_to_intlist,
+    intlist_to_bytes,
+    unified_strdate,
+    url_or_none,
+)
+
+
+class ShemarooMeIE(InfoExtractor):
+    _VALID_URL = r'(?:https?://)(?:www\.)?shemaroome\.com/(?:movies|shows)/(?P<id>[^?#]+)'
+    _TESTS = [{
+        'url': 'https://www.shemaroome.com/movies/dil-hai-tumhaara',
+        'info_dict': {
+            'id': 'dil-hai-tumhaara',
+            'ext': 'mp4',
+            'title': 'Dil Hai Tumhaara',
+            'release_date': '20020906',
+            'thumbnail': 'https://daex9l847wg3n.cloudfront.net/shemoutputimages/Dil-Hai-Tumhaara/60599346a609d2faa3000020/large_16_9_1616436538.jpg?1616483693',
+            'description': 'md5:2782c4127807103cf5a6ae2ca33645ce',
+        },
+        'params': {
+            'skip_download': True
+        }
+    }, {
+        'url': 'https://www.shemaroome.com/shows/jurm-aur-jazbaat/laalach',
+        'info_dict': {
+            'id': 'jurm-aur-jazbaat_laalach',
+            'ext': 'mp4',
+            'title': 'Laalach',
+            'description': 'md5:92b79c2dcb539b0ab53f9fa5a048f53c',
+            'release_date': '20210507',
+        },
+        'params': {
+            'skip_download': True
+        }
+    }]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url).replace('/', '_')
+        webpage = self._download_webpage(url, video_id)
+        m = re.search(
+            r'params_for_player\s*=\s*"(?P<data>[^|]+)\|key=(?P<key>[^|]+)\|image=(?P<thumbnail>[^|]+)\|title=(?P<title>[^|]+)',
+            webpage)
+        data = bytes_to_intlist(compat_b64decode(m.group('data')))
+        key = bytes_to_intlist(compat_b64decode(m.group('key')))
+        iv = [0] * 16
+        m3u8_url = intlist_to_bytes(aes_cbc_decrypt(data, key, iv))
+        m3u8_url = m3u8_url[:-compat_ord((m3u8_url[-1]))].decode('ascii')
+        formats = self._extract_m3u8_formats(m3u8_url, video_id, fatal=False)
+        self._sort_formats(formats)
+
+        release_date = self._html_search_regex(
+            (r'itemprop="uploadDate">\s*([\d-]+)', r'id="release_date" value="([\d-]+)'),
+            webpage, 'release date', fatal=False)
+
+        description = self._html_search_regex(r'(?s)>Synopsis(</.+?)</', webpage, 'description', fatal=False)
+
+        return {
+            'id': video_id,
+            'formats': formats,
+            'title': m.group('title'),
+            'thumbnail': url_or_none(m.group('thumbnail')),
+            'release_date': unified_strdate(release_date),
+            'description': description,
+        }
@@ -9,11 +9,18 @@ from ..compat import compat_HTTPError
 from ..utils import (
    ExtractorError,
    int_or_none,
+    try_get,
 )


 class SonyLIVIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?sonyliv\.com/(?:s(?:how|port)s/[^/]+|movies|clip|trailer|music-videos)/[^/?#&]+-(?P<id>\d+)'
+    _VALID_URL = r'''(?x)
+                     (?:
+                        sonyliv:|
+                        https?://(?:www\.)?sonyliv\.com/(?:s(?:how|port)s/[^/]+|movies|clip|trailer|music-videos)/[^/?#&]+-
+                    )
+                    (?P<id>\d+)
+                  '''
    _TESTS = [{
        'url': 'https://www.sonyliv.com/shows/bachelors-delight-1700000113/achaari-cheese-toast-1000022678?watch=true',
        'info_dict': {
@@ -75,7 +82,7 @@ class SonyLIVIE(InfoExtractor):
        video_id = self._match_id(url)
        content = self._call_api(
            '1.5', 'IN/CONTENT/VIDEOURL/VOD/' + video_id, video_id)
-        if not self._downloader.params.get('allow_unplayable_formats') and content.get('isEncrypted'):
+        if not self.get_param('allow_unplayable_formats') and content.get('isEncrypted'):
            raise ExtractorError('This video is DRM protected.', expected=True)
        dash_url = content['videoURL']
        headers = {
@@ -93,7 +100,14 @@ class SonyLIVIE(InfoExtractor):
        metadata = self._call_api(
            '1.6', 'IN/DETAIL/' + video_id, video_id)['containers'][0]['metadata']
        title = metadata['episodeTitle']
-
+        subtitles = {}
+        for sub in content.get('subtitle', []):
+            sub_url = sub.get('subtitleUrl')
+            if not sub_url:
+                continue
+            subtitles.setdefault(sub.get('subtitleLanguageName', 'ENG'), []).append({
+                'url': sub_url,
+            })
        return {
            'id': video_id,
            'title': title,
@@ -106,4 +120,43 @@ class SonyLIVIE(InfoExtractor):
            'series': metadata.get('title'),
            'episode_number': int_or_none(metadata.get('episodeNumber')),
            'release_year': int_or_none(metadata.get('year')),
+            'subtitles': subtitles,
        }
+
+
+class SonyLIVSeriesIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www\.)?sonyliv\.com/shows/[^/?#&]+-(?P<id>\d{10})$'
+    _TESTS = [{
+        'url': 'https://www.sonyliv.com/shows/adaalat-1700000091',
+        'playlist_mincount': 456,
+        'info_dict': {
+            'id': '1700000091',
+        },
+    }]
+    _API_SHOW_URL = "https://apiv2.sonyliv.com/AGL/1.9/R/ENG/WEB/IN/DL/DETAIL/{}?kids_safe=false&from=0&to=49"
+    _API_EPISODES_URL = "https://apiv2.sonyliv.com/AGL/1.4/R/ENG/WEB/IN/CONTENT/DETAIL/BUNDLE/{}?from=0&to=1000&orderBy=episodeNumber&sortOrder=asc"
+    _API_SECURITY_URL = 'https://apiv2.sonyliv.com/AGL/1.4/A/ENG/WEB/ALL/GETTOKEN'
+
+    def _entries(self, show_id):
+        headers = {
+            'Accept': 'application/json, text/plain, */*',
+            'Referer': 'https://www.sonyliv.com',
+        }
+        headers['security_token'] = self._download_json(
+            self._API_SECURITY_URL, video_id=show_id, headers=headers,
+            note='Downloading security token')['resultObj']
+        seasons = try_get(
+            self._download_json(self._API_SHOW_URL.format(show_id), video_id=show_id, headers=headers),
+            lambda x: x['resultObj']['containers'][0]['containers'], list)
+        for season in seasons or []:
+            season_id = season['id']
+            episodes = try_get(
+                self._download_json(self._API_EPISODES_URL.format(season_id), video_id=season_id, headers=headers),
+                lambda x: x['resultObj']['containers'][0]['containers'], list)
+            for episode in episodes or []:
+                video_id = episode.get('id')
+                yield self.url_result('sonyliv:%s' % video_id, ie=SonyLIVIE.ie_key(), video_id=video_id)
+
+    def _real_extract(self, url):
+        show_id = self._match_id(url)
+        return self.playlist_result(self._entries(show_id), playlist_id=show_id)
@@ -63,7 +63,7 @@ class SportDeutschlandIE(InfoExtractor):
        if len(videos) > 1:
            playlist_id = compat_parse_qs(compat_urllib_parse_urlparse(url).query).get('playlistId', [None])[0]
            if playlist_id:
-                if self._downloader.params.get('noplaylist'):
+                if self.get_param('noplaylist'):
                    videos = [videos[int(playlist_id)]]
                    self.to_screen('Downloading just a single video because of --no-playlist')
                else:
@@ -77,7 +77,7 @@ class SportDeutschlandIE(InfoExtractor):
                        continue
                    formats = self._extract_m3u8_formats(
                        video_url.replace('.smil', '.m3u8'), video_id, 'mp4', fatal=False)
-                    if not formats and not self._downloader.params.get('ignore_no_formats'):
+                    if not formats and not self.get_param('ignore_no_formats'):
                        continue
                    yield {
                        'id': video_id,
@@ -139,7 +139,7 @@ class SteamIE(InfoExtractor):
                                        'format_id': ext + quality,
                                        'url': video_url,
                                    })
-                if not formats and not self._downloader.params.get('ignore_no_formats'):
+                if not formats and not self.get_param('ignore_no_formats'):
                    continue
                entry['formats'] = formats
                entries.append(entry)
@@ -123,6 +123,10 @@ class TEDIE(InfoExtractor):
        'params': {
            'skip_download': True,
        },
+    }, {
+        # with own formats and private Youtube external
+        'url': 'https://www.ted.com/talks/spencer_wells_a_family_tree_for_humanity',
+        'only_matching': True,
    }]

    _NATIVE_FORMATS = {
@@ -210,16 +214,6 @@ class TEDIE(InfoExtractor):

        player_talk = talk_info['player_talks'][0]

-        external = player_talk.get('external')
-        if isinstance(external, dict):
-            service = external.get('service')
-            if isinstance(service, compat_str):
-                ext_url = None
-                if service.lower() == 'youtube':
-                    ext_url = external.get('code')
-
-                return self.url_result(ext_url or external['uri'])
-
        resources_ = player_talk.get('resources') or talk_info.get('resources')

        http_url = None
@@ -294,6 +288,16 @@ class TEDIE(InfoExtractor):
                'vcodec': 'none',
            })

+        if not formats:
+            external = player_talk.get('external')
+            if isinstance(external, dict):
+                service = external.get('service')
+                if isinstance(service, compat_str):
+                    ext_url = None
+                    if service.lower() == 'youtube':
+                        ext_url = external.get('code')
+                    return self.url_result(ext_url or external['uri'])
+
        self._sort_formats(formats)

        video_id = compat_str(talk_info['id'])
@@ -0,0 +1,58 @@
+# coding=utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+from ..utils import (
+    try_get,
+    unified_timestamp,
+    HEADRequest,
+)
+
+
+class TelemundoIE(InfoExtractor):
+
+    _VALID_URL = r'https?:\/\/(?:www\.)?telemundo\.com\/.+?video\/[^\/]+(?P<id>tmvo\d{7})'
+    _TESTS = [{
+        'url': 'https://www.telemundo.com/noticias/noticias-telemundo-en-la-noche/empleo/video/esta-aplicacion-gratuita-esta-ayudando-los-latinos-encontrar-trabajo-en-estados-unidos-tmvo9829325',
+        'info_dict': {
+            'id': 'tmvo9829325',
+            'timestamp': 1621396800,
+            'title': 'Esta aplicación gratuita está ayudando a los latinos a encontrar trabajo en Estados Unidos',
+            'uploader': 'Telemundo',
+            'uploader_id': 'NBCU_Telemundo',
+            'ext': 'mp4',
+            'upload_date': '20210519',
+        },
+        'params': {
+            'skip_download': True,
+        }
+    }, {
+        'url': 'https://www.telemundo.com/shows/al-rojo-vivo/empleo/video/personajes-de-times-square-piden-que-la-ciudad-de-nueva-york-los-deje-volver-trabajar-tmvo9816272',
+        'only_matching': True,
+    }]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        webpage = self._download_webpage(url, video_id)
+        metadata = self._parse_json(
+            self._search_regex(r'<[^>]+id="__NEXT_DATA__"[^>]+>([^<]+)', webpage, 'JSON metadata'), video_id)
+        redirect_url = try_get(
+            metadata,
+            lambda x: x['props']['initialState']['video']['associatedPlaylists'][0]['videos'][0]['videoAssets'][0]['publicUrl'])
+
+        m3u8_url = self._request_webpage(HEADRequest(
+            redirect_url + '?format=redirect&manifest=m3u&format=redirect&Tracking=true&Embedded=true&formats=MPEG4'),
+            video_id, 'Processing m3u8').geturl()
+        formats = self._extract_m3u8_formats(m3u8_url, video_id, 'mp4')
+        self._sort_formats(formats)
+        date = unified_timestamp(try_get(
+            metadata, lambda x: x['props']['initialState']['video']['associatedPlaylists'][0]['videos'][0]['datePublished'].split(' ', 1)[1]))
+        return {
+            'url': url,
+            'id': video_id,
+            'title': self._search_regex(r'<h1[^>]+>([^<]+)', webpage, 'title', fatal=False),
+            'formats': formats,
+            'timestamp': date,
+            'uploader': 'Telemundo',
+            'uploader_id': self._search_regex(r'https?:\/\/(?:[^/]+\/){3}video\/(?P<id>[^\/]+)', m3u8_url, 'Akamai account', fatal=False)
+        }
@@ -1,70 +1,89 @@
 # coding: utf-8
 from __future__ import unicode_literals

+from datetime import datetime
+import base64
+
 from .common import InfoExtractor
 from ..utils import (
    HEADRequest,
-    parse_age_limit,
-    parse_iso8601,
-    # smuggle_url,
+    urlencode_postdata,
 )


 class TenPlayIE(InfoExtractor):
    _VALID_URL = r'https?://(?:www\.)?10play\.com\.au/(?:[^/]+/)+(?P<id>tpv\d{6}[a-z]{5})'
+    _NETRC_MACHINE = '10play'
    _TESTS = [{
-        'url': 'https://10play.com.au/masterchef/episodes/season-1/masterchef-s1-ep-1/tpv190718kwzga',
+        'url': 'https://10play.com.au/todd-sampsons-body-hack/episodes/season-4/episode-7/tpv200921kvngh',
        'info_dict': {
-            'id': '6060533435001',
+            'id': '6192880312001',
            'ext': 'mp4',
-            'title': 'MasterChef - S1 Ep. 1',
-            'description': 'md5:4fe7b78e28af8f2d900cd20d900ef95c',
-            'age_limit': 10,
-            'timestamp': 1240828200,
-            'upload_date': '20090427',
-            'uploader_id': '2199827728001',
+            'title': "Todd Sampson's Body Hack - S4 Ep. 2",
+            'description': 'md5:fa278820ad90f08ea187f9458316ac74',
+            'age_limit': 15,
+            'timestamp': 1600770600,
+            'upload_date': '20200922',
+            'uploader': 'Channel 10',
+            'uploader_id': '2199827728001'
        },
        'params': {
-            # 'format': 'bestvideo',
            'skip_download': True,
        }
    }, {
        'url': 'https://10play.com.au/how-to-stay-married/web-extras/season-1/terrys-talks-ep-1-embracing-change/tpv190915ylupc',
        'only_matching': True,
    }]
-    # BRIGHTCOVE_URL_TEMPLATE = 'https://players.brightcove.net/2199827728001/cN6vRtRQt_default/index.html?videoId=%s'
    _GEO_BYPASS = False
-    _FASTLY_URL_TEMPL = 'https://10-selector.global.ssl.fastly.net/s/kYEXFC/media/%s?mbr=true&manifest=m3u&format=redirect'
+
+    _AUS_AGES = {
+        'G': 0,
+        'PG': 15,
+        'M': 15,
+        'MA': 15,
+        'R': 18,
+        'X': 18
+    }
+
+    def _get_bearer_token(self, video_id):
+        username, password = self._get_login_info()
+        if username is None or password is None:
+            self.raise_login_required('Your 10play account\'s details must be provided with --username and --password.')
+        _timestamp = datetime.now().strftime('%Y%m%d000000')
+        _auth_header = base64.b64encode(_timestamp.encode('ascii')).decode('ascii')
+        data = self._download_json('https://10play.com.au/api/user/auth', video_id, 'Getting bearer token', headers={
+            'X-Network-Ten-Auth': _auth_header,
+        }, data=urlencode_postdata({
+            'email': username,
+            'password': password,
+        }))
+        return "Bearer " + data['jwt']['accessToken']

    def _real_extract(self, url):
        content_id = self._match_id(url)
+        _token = self._get_bearer_token(content_id)
        data = self._download_json(
-            'https://10play.com.au/api/video/' + content_id, content_id)
-        video = data.get('video') or {}
-        metadata = data.get('metaData') or {}
-        brightcove_id = video.get('videoId') or metadata['showContentVideoId']
-        # brightcove_url = smuggle_url(
-        #     self.BRIGHTCOVE_URL_TEMPLATE % brightcove_id,
-        #     {'geo_countries': ['AU']})
+            'https://10play.com.au/api/v1/videos/' + content_id, content_id)
+        _video_url = self._download_json(
+            data.get('playbackApiEndpoint'), content_id, 'Downloading video JSON',
+            headers={'Authorization': _token}).get('source')
        m3u8_url = self._request_webpage(HEADRequest(
-            self._FASTLY_URL_TEMPL % brightcove_id), brightcove_id).geturl()
+            _video_url), content_id).geturl()
        if '10play-not-in-oz' in m3u8_url:
            self.raise_geo_restricted(countries=['AU'])
-        formats = self._extract_m3u8_formats(m3u8_url, brightcove_id, 'mp4')
+        formats = self._extract_m3u8_formats(m3u8_url, content_id, 'mp4')
        self._sort_formats(formats)

        return {
-            # '_type': 'url_transparent',
-            # 'url': brightcove_url,
            'formats': formats,
-            'id': brightcove_id,
-            'title': video.get('title') or metadata.get('pageContentName') or metadata['showContentName'],
-            'description': video.get('description'),
-            'age_limit': parse_age_limit(video.get('showRatingClassification') or metadata.get('showProgramClassification')),
-            'series': metadata.get('showName'),
-            'season': metadata.get('showContentSeason'),
-            'timestamp': parse_iso8601(metadata.get('contentPublishDate') or metadata.get('pageContentPublishDate')),
-            'thumbnail': video.get('poster'),
+            'id': data.get('altId') or content_id,
+            'title': data.get('title'),
+            'description': data.get('description'),
+            'age_limit': self._AUS_AGES[data.get('classification')],
+            'series': data.get('showName'),
+            'season': data.get('showContentSeason'),
+            'timestamp': data.get('published'),
+            'thumbnail': data.get('imageUrl'),
+            'uploader': 'Channel 10',
            'uploader_id': '2199827728001',
-            # 'ie_key': 'BrightcoveNew',
        }
@@ -153,7 +153,7 @@ class ToggleIE(InfoExtractor):
                })
        if not formats:
            for meta in (info.get('Metas') or []):
-                if (not self._downloader.params.get('allow_unplayable_formats')
+                if (not self.get_param('allow_unplayable_formats')
                        and meta.get('Key') == 'Encryption' and meta.get('Value') == '1'):
                    self.raise_no_formats(
                        'This video is DRM protected.', expected=True)
@@ -74,7 +74,7 @@ class TouTvIE(RadioCanadaIE):
            })
        # IsDrm does not necessarily mean the video is DRM protected (see
        # https://github.com/ytdl-org/youtube-dl/issues/13994).
-        if not self._downloader.params.get('allow_unplayable_formats') and metadata.get('IsDrm'):
+        if not self.get_param('allow_unplayable_formats') and metadata.get('IsDrm'):
            self.report_warning('This video is probably DRM protected.', path)
        video_id = metadata['IdMedia']
        details = metadata['Details']
@@ -69,7 +69,7 @@ class TVNowBaseIE(InfoExtractor):
            if formats:
                break
        else:
-            if not self._downloader.params.get('allow_unplayable_formats') and info.get('isDrm'):
+            if not self.get_param('allow_unplayable_formats') and info.get('isDrm'):
                raise ExtractorError(
                    'Video %s is DRM protected' % video_id, expected=True)
            if info.get('geoblocked'):
@@ -59,7 +59,7 @@ class TwitCastingIE(InfoExtractor):
    def _real_extract(self, url):
        uploader_id, video_id = re.match(self._VALID_URL, url).groups()

-        video_password = self._downloader.params.get('videopassword')
+        video_password = self.get_param('videopassword')
        request_data = None
        if video_password:
            request_data = urlencode_postdata({
@@ -49,6 +49,7 @@ class TwitchBaseIE(InfoExtractor):
        'ChannelCollectionsContent': '07e3691a1bad77a36aba590c351180439a40baefc1c275356f40fc7082419a84',
        'StreamMetadata': '1c719a40e481453e5c48d9bb585d971b8b372f8ebb105b17076722264dfa5b3e',
        'ComscoreStreamingQuery': 'e1edae8122517d013405f237ffcc124515dc6ded82480a88daef69c83b53ac01',
+        'VideoAccessToken_Clip': '36b89d2507fce29e5ca551df756d27c1cfe079e2609642b4390aa4c35796eb11',
        'VideoPreviewOverlay': '3006e77e51b128d838fa4e835723ca4dc9a05c5efd4466c1085215c6e437e65c',
        'VideoMetadata': '226edb3e692509f727fd56821f5653c05740242c82b0388883e0c0e75dcbf687',
    }
@@ -893,7 +894,25 @@ class TwitchClipsIE(TwitchBaseIE):
    def _real_extract(self, url):
        video_id = self._match_id(url)

-        clip = self._download_base_gql(
+        clip = self._download_gql(
+            video_id, [{
+                'operationName': 'VideoAccessToken_Clip',
+                'variables': {
+                    'slug': video_id,
+                },
+            }],
+            'Downloading clip access token GraphQL')[0]['data']['clip']
+
+        if not clip:
+            raise ExtractorError(
+                'This clip is no longer available', expected=True)
+
+        access_query = {
+            'sig': clip['playbackAccessToken']['signature'],
+            'token': clip['playbackAccessToken']['value'],
+        }
+
+        data = self._download_base_gql(
            video_id, {
                'query': '''{
  clip(slug: "%s") {
@@ -918,11 +937,10 @@ class TwitchClipsIE(TwitchBaseIE):
    }
    viewCount
  }
-}''' % video_id}, 'Downloading clip GraphQL')['data']['clip']
+}''' % video_id}, 'Downloading clip GraphQL', fatal=False)

-        if not clip:
-            raise ExtractorError(
-                'This clip is no longer available', expected=True)
+        if data:
+            clip = try_get(data, lambda x: x['data']['clip'], dict) or clip

        formats = []
        for option in clip.get('videoQualities', []):
@@ -932,7 +950,7 @@ class TwitchClipsIE(TwitchBaseIE):
            if not source:
                continue
            formats.append({
-                'url': source,
+                'url': update_url_query(source, access_query),
                'format_id': option.get('quality'),
                'height': int_or_none(option.get('quality')),
                'fps': int_or_none(option.get('frameRate')),
@@ -75,7 +75,7 @@ class UstreamIE(InfoExtractor):
    @staticmethod
    def _extract_url(webpage):
        mobj = re.search(
-            r'<iframe[^>]+?src=(["\'])(?P<url>http://(?:www\.)?(?:ustream\.tv|video\.ibm\.com)/embed/.+?)\1', webpage)
+            r'<iframe[^>]+?src=(["\'])(?P<url>https?://(?:www\.)?(?:ustream\.tv|video\.ibm\.com)/embed/.+?)\1', webpage)
        if mobj is not None:
            return mobj.group('url')

@@ -5,11 +5,14 @@ import re

 from .common import InfoExtractor
 from ..utils import (
+    ExtractorError,
+    get_element_by_class,
    int_or_none,
    parse_iso8601,
    str_or_none,
    strip_or_none,
    try_get,
+    urlencode_postdata,
 )


@@ -41,11 +44,49 @@ class VidioIE(InfoExtractor):
    }, {
        'url': 'https://www.vidio.com/watch/77949-south-korea-test-fires-missile-that-can-strike-all-of-the-north',
        'only_matching': True,
+    }, {
+        # Premier-exclusive video
+        'url': 'https://www.vidio.com/watch/1550718-stand-by-me-doraemon',
+        'only_matching': True
    }]
+    _LOGIN_URL = 'https://www.vidio.com/users/login'
+    _NETRC_MACHINE = 'vidio'
+
+    def _login(self):
+        username, password = self._get_login_info()
+        if username is None:
+            return
+
+        def is_logged_in():
+            res = self._download_json(
+                'https://www.vidio.com/interactions.json', None, 'Checking if logged in', fatal=False) or {}
+            return bool(res.get('current_user'))
+
+        if is_logged_in():
+            return
+
+        login_page = self._download_webpage(
+            self._LOGIN_URL, None, 'Downloading log in page')
+
+        login_form = self._form_hidden_inputs("login-form", login_page)
+        login_form.update({
+            'user[login]': username,
+            'user[password]': password,
+        })
+        login_post, login_post_urlh = self._download_webpage_handle(
+            self._LOGIN_URL, None, 'Logging in', data=urlencode_postdata(login_form), expected_status=[302, 401])
+
+        if login_post_urlh.status == 401:
+            reason = get_element_by_class('onboarding-form__general-error', login_post)
+            if reason:
+                raise ExtractorError(
+                    'Unable to log in: %s' % reason, expected=True)
+            raise ExtractorError('Unable to log in')

    def _real_initialize(self):
        self._api_key = self._download_json(
            'https://www.vidio.com/auth', None, data=b'')['api_key']
+        self._login()

    def _real_extract(self, url):
        video_id, display_id = re.match(self._VALID_URL, url).groups()
@@ -56,9 +97,30 @@ class VidioIE(InfoExtractor):
            })
        video = data['videos'][0]
        title = video['title'].strip()
+        is_premium = video.get('is_premium')
+        if is_premium:
+            sources = self._download_json(
+                'https://www.vidio.com/interactions_stream.json?video_id=%s&type=videos' % video_id,
+                display_id, note='Downloading premier API JSON')
+            if not (sources.get('source') or sources.get('source_dash')):
+                self.raise_login_required('This video is only available for registered users with a premier subscription.')
+
+            formats, subs = [], {}
+            if sources.get('source'):
+                hls_formats, hls_subs = self._extract_m3u8_formats_and_subtitles(
+                    sources['source'], display_id, 'mp4', 'm3u8_native')
+                formats.extend(hls_formats)
+                subs.update(hls_subs)
+            if sources.get('source_dash'):  # TODO: Find video example with source_dash
+                dash_formats, dash_subs = self._extract_mpd_formats_and_subtitles(
+                    sources['source_dash'], display_id, 'dash')
+                formats.extend(dash_formats)
+                subs.update(dash_subs)
+        else:
+            hls_url = data['clips'][0]['hls_url']
+            formats, subs = self._extract_m3u8_formats_and_subtitles(
+                hls_url, display_id, 'mp4', 'm3u8_native')

-        formats = self._extract_m3u8_formats(
-            data['clips'][0]['hls_url'], display_id, 'mp4', 'm3u8_native')
        self._sort_formats(formats)

        get_first = lambda x: try_get(data, lambda y: y[x + 's'][0], dict) or {}
@@ -76,6 +138,7 @@ class VidioIE(InfoExtractor):
            'duration': int_or_none(video.get('duration')),
            'like_count': get_count('likes'),
            'formats': formats,
+            'subtitles': subs,
            'uploader': user.get('name'),
            'timestamp': parse_iso8601(video.get('created_at')),
            'uploader_id': username,
@@ -324,7 +324,7 @@ class VikiIE(VikiBaseIE):
                # Despite CODECS metadata in m3u8 all video-only formats
                # are actually video+audio
                for f in m3u8_formats:
-                    if not self._downloader.params.get('allow_unplayable_formats') and '_drm/index_' in f['url']:
+                    if not self.get_param('allow_unplayable_formats') and '_drm/index_' in f['url']:
                        continue
                    if f.get('acodec') == 'none' and f.get('vcodec') != 'none':
                        f['acodec'] = None
@@ -76,7 +76,7 @@ class VimeoBaseInfoExtractor(InfoExtractor):
            raise ExtractorError('Unable to log in')

    def _get_video_password(self):
-        password = self._downloader.params.get('videopassword')
+        password = self.get_param('videopassword')
        if password is None:
            raise ExtractorError(
                'This video is protected by a password, use the --video-password option',
@@ -603,7 +603,7 @@ class VimeoIE(VimeoBaseInfoExtractor):
            album_id, headers={'Authorization': 'jwt ' + jwt},
            query={'fields': 'description,name,privacy'})
        if try_get(album, lambda x: x['privacy']['view']) == 'password':
-            password = self._downloader.params.get('videopassword')
+            password = self.get_param('videopassword')
            if not password:
                raise ExtractorError(
                    'This album is protected by a password, use the --video-password option',
@@ -1058,7 +1058,7 @@ class VimeoAlbumIE(VimeoBaseInfoExtractor):
            query={'fields': 'description,name,privacy'})
        hashed_pass = None
        if try_get(album, lambda x: x['privacy']['view']) == 'password':
-            password = self._downloader.params.get('videopassword')
+            password = self.get_param('videopassword')
            if not password:
                raise ExtractorError(
                    'This album is protected by a password, use the --video-password option',
@@ -287,7 +287,7 @@ class ViuOTTIE(InfoExtractor):
            raise ExtractorError('This video is not available in your region.', expected=True)

        series_id = video_data.get('series_id')
-        if not self._downloader.params.get('noplaylist') and not idata.get('force_noplaylist'):
+        if not self.get_param('noplaylist') and not idata.get('force_noplaylist'):
            self.to_screen('Downloading playlist %s - add --no-playlist to just download video' % series_id)
            series = product_data.get('series', {})
            product = series.get('product')
@@ -308,7 +308,7 @@ class ViuOTTIE(InfoExtractor):

                return self.playlist_result(entries, series_id, series.get('name'), series.get('description'))

-        if self._downloader.params.get('noplaylist'):
+        if self.get_param('noplaylist'):
            self.to_screen('Downloading just video %s because of --no-playlist' % video_id)

        duration_limit = False
@@ -300,6 +300,13 @@ class VKIE(VKBaseIE):
            'only_matching': True,
        }]

+    @staticmethod
+    def _extract_sibnet_urls(webpage):
+        # https://help.sibnet.ru/?sibnet_video_embed
+        return [unescapeHTML(mobj.group('url')) for mobj in re.finditer(
+            r'<iframe\b[^>]+\bsrc=(["\'])(?P<url>(?:https?:)?//video\.sibnet\.ru/shell\.php\?.*?\bvideoid=\d+.*?)\1',
+            webpage)]
+
    def _real_extract(self, url):
        mobj = re.match(self._VALID_URL, url)
        video_id = mobj.group('videoid')
@@ -408,6 +415,10 @@ class VKIE(VKBaseIE):
        if odnoklassniki_url:
            return self.url_result(odnoklassniki_url, OdnoklassnikiIE.ie_key())

+        sibnet_urls = self._extract_sibnet_urls(info_page)
+        if sibnet_urls:
+            return self.url_result(sibnet_urls[0])
+
        m_opts = re.search(r'(?s)var\s+opts\s*=\s*({.+?});', info_page)
        if m_opts:
            m_opts_url = re.search(r"url\s*:\s*'((?!/\b)[^']+)", m_opts.group(1))
@@ -136,7 +136,7 @@ class VLiveIE(VLiveBaseIE):
            'author{nickname},channel{channelCode,channelName},officialVideo{commentCount,exposeStatus,likeCount,playCount,playTime,status,title,type,vodId},playlist{playlistSeq,totalCount,name}')

        playlist = post.get('playlist')
-        if not playlist or self._downloader.params.get('noplaylist'):
+        if not playlist or self.get_param('noplaylist'):
            if playlist:
                self.to_screen(
                    'Downloading just video %s because of --no-playlist'
@@ -2,6 +2,7 @@
 from __future__ import unicode_literals

 from .common import InfoExtractor
+from ..compat import compat_str
 from ..utils import (
    ExtractorError,
    int_or_none,
@@ -11,7 +12,17 @@ from ..utils import (


 class VootIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?voot\.com/(?:[^/]+/)+(?P<id>\d+)'
+    _VALID_URL = r'''(?x)
+                    (?:
+                        voot:|
+                        (?:https?://)(?:www\.)?voot\.com/?
+                        (?:
+                            movies/[^/]+/|
+                            (?:shows|kids)/(?:[^/]+/){4}
+                        )
+                     )
+                    (?P<id>\d{3,})
+                    '''
    _GEO_COUNTRIES = ['IN']
    _TESTS = [{
        'url': 'https://www.voot.com/shows/ishq-ka-rang-safed/1/360558/is-this-the-end-of-kamini-/441353',
@@ -22,7 +33,6 @@ class VootIE(InfoExtractor):
            'description': 'md5:06291fbbbc4dcbe21235c40c262507c1',
            'timestamp': 1472162937,
            'upload_date': '20160825',
-            'duration': 1146,
            'series': 'Ishq Ka Rang Safed',
            'season_number': 1,
            'episode': 'Is this the end of Kamini?',
@@ -44,7 +54,6 @@ class VootIE(InfoExtractor):

    def _real_extract(self, url):
        video_id = self._match_id(url)
-
        media_info = self._download_json(
            'https://wapi.voot.com/ws/ott/getMediaInfo.json', video_id,
            query={
@@ -82,7 +91,6 @@ class VootIE(InfoExtractor):
                episode = value
            elif key == 'EpisodeNo':
                episode_number = int_or_none(value)
-
        return {
            'extractor_key': 'Kaltura',
            'id': entry_id,
@@ -98,3 +106,45 @@ class VootIE(InfoExtractor):
            'like_count': int_or_none(media.get('like_counter')),
            'formats': formats,
        }
+
+
+class VootSeriesIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www\.)?voot\.com/shows/[^/]+/(?P<id>\d{3,})'
+    _TESTS = [{
+        'url': 'https://www.voot.com/shows/chakravartin-ashoka-samrat/100002',
+        'playlist_mincount': 442,
+        'info_dict': {
+            'id': '100002',
+        },
+    }, {
+        'url': 'https://www.voot.com/shows/ishq-ka-rang-safed/100003',
+        'playlist_mincount': 341,
+        'info_dict': {
+            'id': '100003',
+        },
+    }]
+    _SHOW_API = 'https://psapi.voot.com/media/voot/v1/voot-web/content/generic/season-by-show?sort=season%3Aasc&id={}&responseType=common'
+    _SEASON_API = 'https://psapi.voot.com/media/voot/v1/voot-web/content/generic/series-wise-episode?sort=episode%3Aasc&id={}&responseType=common&page={:d}'
+
+    def _entries(self, show_id):
+        show_json = self._download_json(self._SHOW_API.format(show_id), video_id=show_id)
+        for season in show_json.get('result', []):
+            page_num = 1
+            season_id = try_get(season, lambda x: x['id'], compat_str)
+            season_json = self._download_json(self._SEASON_API.format(season_id, page_num),
+                                              video_id=season_id,
+                                              note='Downloading JSON metadata page %d' % page_num)
+            episodes_json = season_json.get('result', [])
+            while episodes_json:
+                page_num += 1
+                for episode in episodes_json:
+                    video_id = episode.get('id')
+                    yield self.url_result(
+                        'voot:%s' % video_id, ie=VootIE.ie_key(), video_id=video_id)
+                episodes_json = self._download_json(self._SEASON_API.format(season_id, page_num),
+                                                    video_id=season_id,
+                                                    note='Downloading JSON metadata page %d' % page_num)['result']
+
+    def _real_extract(self, url):
+        show_id = self._match_id(url)
+        return self.playlist_result(self._entries(show_id), playlist_id=show_id)
@@ -41,7 +41,7 @@ class WakanimIE(InfoExtractor):
        m3u8_url = urljoin(url, self._search_regex(
            r'file\s*:\s*(["\'])(?P<url>(?:(?!\1).)+)\1', webpage, 'm3u8 url',
            group='url'))
-        if not self._downloader.params.get('allow_unplayable_formats'):
+        if not self.get_param('allow_unplayable_formats'):
            # https://docs.microsoft.com/en-us/azure/media-services/previous/media-services-content-protection-overview#streaming-urls
            encryption = self._search_regex(
                r'encryption%3D(c(?:enc|bc(?:s-aapl)?))',
@@ -47,8 +47,7 @@ class WhoWatchIE(InfoExtractor):

            if hls_url:
                hls_fmts = self._extract_m3u8_formats(
-                    hls_url, video_id, ext='mp4', entry_protocol='m3u8',
-                    m3u8_id='hls-%s' % name, quality=quality)
+                    hls_url, video_id, ext='mp4', m3u8_id='hls-%s' % name, quality=quality)
                formats.extend(hls_fmts)
            else:
                hls_fmts = []
@@ -71,8 +70,7 @@ class WhoWatchIE(InfoExtractor):

        # This contains the same formats as the above manifests and is used only as a fallback
        formats.extend(self._extract_m3u8_formats(
-            hls_url, video_id, ext='mp4', entry_protocol='m3u8',
-            m3u8_id='hls'))
+            hls_url, video_id, ext='mp4', m3u8_id='hls'))
        self._remove_duplicate_formats(formats)
        self._sort_formats(formats)

--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
pukkandan	3de7c2ce9a	Release 2021.06.01	2021-06-01 20:29:03 +05:30
pukkandan	bc6b9bcd65	[utils] Escape URLs in `sanitized_Request`, not `sanitize_url` `d2558234cf` added escaping of URLs while sanitizing. However, `sanitize_url` may not always receive an actual URL. Eg: When using `yt-dlp "search query" --default-search ytsearch`, `search query` gets escaped to `search%20query` before being prefixed with `ytsearch:` which is not the intended behavior. So the escaping is moved to `sanitized_Request` instead.	2021-06-01 20:29:02 +05:30
AshishandGitHub	6e6390321c	[Hotstar] Add HotStarSeriesIE (#366 ) Authored by: Ashish0804	2021-06-01 20:14:03 +05:30
pukkandan	4040428efc	[update] Block further update for unsupported systems	2021-06-01 03:32:09 +05:30
pukkandan	cc1dfc9373	[cleanup] setup.py	2021-06-01 02:48:20 +05:30
pukkandan	14eb1ee1cb	Update to ytdl-commit-d495292 [ard] Relax _VALID_URL and fix video ids https://github.com/ytdl-org/youtube-dl/commit/d495292852b6c2f1bd58bc2141ff2b0265c952cf Closes #357	2021-06-01 02:48:20 +05:30
coletdjnzandGitHub	879e7199bb	[archiveorg] Add YoutubeWebArchiveIE (#356 ) Co-authored by: colethedj, pukkandan, alex-gedeon	2021-05-31 01:12:38 +00:00
pukkandan	d89da64b1d	[hls,dash] When using `concurrent_fragment_downloads`, do not keep the fragment content in memory Partial fix for #359 This is a temporary solution until #364 can be implemented	2021-05-30 23:01:49 +05:30
pukkandan	5dcd8e1d88	[hls] Disable external downloader for `webtt`	2021-05-30 23:00:57 +05:30
MinePlayersPEandGitHub	10bb7e51e8	[vidio] Add login support (#362 ) Authored by: MinePlayersPE	2021-05-30 20:19:14 +05:30
pukkandan	b0089e8992	[fancode] Add extractor (#316,#354) Closes #269, #363 Authored by: rmsmachine	2021-05-30 19:34:39 +05:30
AshishandGitHub	a3ed14cbaf	[Voot] Add VootSeriesIE (#351 ) Authored by: Ashish0804	2021-05-30 16:45:42 +05:30
LEandGitHub	9dee4df559	[Saitosan] Add new extractor (#350 ) Closes #224 Authored by: llacb47	2021-05-30 14:32:18 +05:30
pukkandan	adddc50cbf	[extractor] Functions to parse socket.io response as json Authored by: pukkandan, llacb47	2021-05-30 14:22:42 +05:30
MinePlayersPEandGitHub	46c43ffc9d	[vidio] Support premium videos (#358 ) Authored by: MinePlayersPE	2021-05-29 20:24:19 +05:30
pukkandan	37a3bb66a7	[extractor] Allow `note=False` when extracting manifests	2021-05-29 14:22:44 +05:30
pukkandan	337e0c62f8	[embedthumbnail] Correctly escape filename Closes #352 The approach in [1] is faulty as can be seen in the test cases 1. https://github.com/ytdl-org/youtube-dl/commit/bff857a8af696e701482208617bf0b7564951326	2021-05-29 02:31:14 +05:30
pukkandan	885cc0b75c	[embedthumbnail] Embed if any thumbnail was downloaded, not just the best	2021-05-29 02:31:14 +05:30
pukkandan	46953e7e6e	[youtube:playlist] fix bug	2021-05-29 02:31:13 +05:30
pukkandan	ae8f99e648	Remove `None` values from `info.json`	2021-05-29 02:31:13 +05:30
pukkandan	077c476276	[zee5] Fix m3u8 formats extension	2021-05-29 02:31:12 +05:30
pukkandan	835a1478b4	Write messages to `stderr` when both `quiet` and `verbose`	2021-05-29 02:31:10 +05:30
pukkandan	120fe5134a	Pre-check archive and filters during playlist extraction This makes `--break-on-existing` much faster. It also helps `--break-on-reject` if the playlist extractor can extract the relevant fields	2021-05-29 02:12:09 +05:30
pukkandan	56a8fb4f77	Refactor `__process_playlist` using `LazyList`	2021-05-29 02:12:09 +05:30
pukkandan	55575225b4	[utils] Add `__getitem__` for `PagedList`	2021-05-29 02:12:08 +05:30
pukkandan	483336e79e	[utils] Add `LazyList`	2021-05-29 02:12:08 +05:30
pukkandan	c77495e3a4	[cleanup] `_match_entry`	2021-05-29 02:12:07 +05:30
rhsmachineandGitHub	65af1839c6	[patreon] Support vimeo embeds (#349 ) Authored by: rhsmachine	2021-05-27 22:14:43 +05:30
pukkandan	177877c544	[extractor] Always prefer native hls downloader by default When the manifest is not downloadable by native downloader, it already is able to detect it and switch to `ffmpeg`. So there doesn't seem to be a reason anymore to use ffmpeg as the preferred downloader	2021-05-26 01:27:39 +05:30
pukkandan	b25522ba52	[update] Replace self without launching a subprocess in windows Closes: #335, https://github.com/ytdl-org/youtube-dl/issues/28488, https://github.com/ytdl-org/youtube-dl/issues/5810, https://github.com/ytdl-org/youtube-dl/issues/5994 In windows, a running executable cannot be replaced. So, the old updater worked by launching a batch script and then exiting, so that the batch script can replace the executable. However, this caused the above-mentioned issues. The new method takes advantage of the fact that while the executable cannot be replaced or deleted, it can still be renamed. The current update process on windows is as follows: 1. Delete `yt-dlp.exe.old` if it exists 2. Download the new version as `yt-dlp.exe.new` 3. Rename the running exe to `yt-dlp.exe.old` 4. Rename `yt-dlp.exe.new` to `yt-dlp.exe` 5. Open a shell that deletes `yt-dlp.exe.old` and terminate While we still use a subprocess, the actual update is already done before the app terminates and the batch script does not print anything to stdout/stderr. So this solves all the above issues	2021-05-26 01:13:34 +05:30
pukkandan	c19bc311cb	[cleanup] Refactor updater The updater now uses `.update.run_update` and not `.update.update_self`. Although I don't expect anyone to be using the updater via API, a wrapper `update_self` is provided for compatibility just in case	2021-05-26 01:13:08 +05:30
Hubert Hirtzandpukkandan	5435dcf96e	Handle Basic Auth `user:pass` in URLs Fixes https://github.com/ytdl-org/youtube-dl/issues/20258, https://github.com/ytdl-org/youtube-dl/issues/26211 Authored by: hhirtz, pukkandan	2021-05-24 03:38:02 +05:30
Oliver Freyermuthandpukkandan	f17c702270	[ard] Allow URLs without `-` before id https://github.com/ytdl-org/youtube-dl/pull/29091 Authored by: olifre	2021-05-23 23:03:08 +05:30
pukkandan	3907333c5d	[extractor] Skip subtitles without URI in m3u8 manifests Closes #339 Authored by: hheimbuerger	2021-05-23 22:32:47 +05:30
pukkandan	acdecdfaef	[embedthumbnail] Embed in `mp4`/`m4a` using mutagen Code from: https://github.com/ytdl-org/youtube-dl/pull/23525 Co-authored by: tripulse , pukkandan	2021-05-23 22:16:35 +05:30
AshishandGitHub	09d18ad07e	[Sonyliv] Add subtitle support (#342 ) Authored by: Ashish0804	2021-05-23 21:26:27 +05:30
pukkandan	bc516a3f3c	Sanitize and sort playlist thumbnails Closes #341	2021-05-23 17:28:15 +05:30
AshishandGitHub	9572eaaa11	[ShemarooMe] Add extractor (#332 ) Closes #307 Co-authored-by: Ashish0804, pukkandan	2021-05-23 12:42:50 +05:30
pukkandan	18e674b4f6	[ffmpeg] Download and merge in a single step if possible	2021-05-23 03:53:18 +05:30
pukkandan	8d68ab98a7	[youtube] Fix bug where not all hls formats were extracted Bug introduced in `9297939ec3`	2021-05-23 03:53:17 +05:30
AshishandGitHub	135e6b93f4	[SonyLIV] Add SonyLIVSeriesIE (#331 ) Authored by: Ashish0804	2021-05-22 17:53:06 +05:30
king-millezandGitHub	13a49340ed	[telemundo] add extractor (#327 ) Closes #284 Authored by: king-millez	2021-05-22 17:17:49 +05:30
pukkandan	81a23040eb	[cleanup] Refactor ffmpeg convertors	2021-05-22 15:20:42 +05:30
pukkandan	857f63136d	[videoconvertor] Generalize with remuxer and allow conditional recoding	2021-05-22 15:20:42 +05:30
louie-githubandGitHub	a927acb1ec	[ThumbnailsConvertor] Support conversion to `png` and make it the default (#333 ) PNG, being a lossless format, should be a better default here compared to JPG since we won't be compressing to a lossy format and losing some of the original image data PNG is also supported for embedding in all the formats similar to JPEG Authored by: louie-github	2021-05-21 23:39:48 +05:30
pukkandan	09f1580e2d	[youtube] `/live` URLs should raise error if channel is not live Fixes: https://github.com/ytdl-org/youtube-dl/issues/29090	2021-05-21 20:05:54 +05:30
pukkandan	cd59e22191	[version] update :ci skip all	2021-05-20 21:15:41 +05:30
shirtandpukkandan	7237fdc6ce	[build] Fix `pefile` version for x86 Authored by: shirt-dev	2021-05-20 21:15:41 +05:30
pukkandan	0fdf490d33	Release 2021.05.20	2021-05-20 21:13:19 +05:30
pukkandan	b73612a254	Update to ytdl-commit-dfbbe29 [redbulltv] fix embed data extraction https://github.com/ytdl-org/youtube-dl/commit/dfbbe2902fc67f0f93ee47a8077c148055c67a9b	2021-05-20 21:13:18 +05:30
king-millezandGitHub	5014558ab9	[parlview] Add extractor (#322 ) Authored by: king-millez	2021-05-20 18:35:37 +05:30
pukkandan	28b0eb0f65	[cleanup] See desc * Remove struct from `embedthumbnail` * Use bullet lists in readme where numbered list don't make sense * Fix error introduced in `9c2b75b561` when `ie_result` is `None`	2021-05-20 18:02:58 +05:30
pukkandan	95131b2176	[embedthumbnail] Add `flac` support and refactor `mutagen` code https://github.com/ytdl-org/youtube-dl/pull/28894, https://github.com/ytdl-org/youtube-dl/pull/24310 Authored by: tripulse	2021-05-20 17:51:33 +05:30
pukkandan	2305e2e5c9	[options] Alias `--write-comments`, `--no-write-comments` Closes: #264	2021-05-20 15:56:57 +05:30
coletdjnzandpukkandan	00ae27690d	[youtube] Add `html5=1` param to `get_video_info` page requests (#329 ) Workaround for #319, https://github.com/ytdl-org/youtube-dl/issues/29086 Authored by: colethedj	2021-05-20 15:56:57 +05:30
pukkandan	9d5d4d64f8	[youtube] Better message when login required	2021-05-20 15:55:55 +05:30
king-millezandpukkandan	98784ef8d6	[audius:artist] Add extractor (#323 ) Authored by: king-millez	2021-05-20 15:55:55 +05:30
pukkandan	d3fc8074a4	[youtube] Sort audio-only formats correctly Closes #317	2021-05-19 18:29:20 +05:30
pukkandan	9c2b75b561	Field `additional_urls` to download additional videos from metadata	2021-05-19 18:11:15 +05:30
pukkandan	856bb8f99d	[downloader] Fix `write_debug`	2021-05-19 17:34:17 +05:30
pukkandan	af32f40bf5	[test] Fix `test_YoutubeDL.TestYoutubeDL` Test `test_ignoreerrors_for_playlist_with_url_transparent_iterable_entries` was broken due to `__original_infodict` being added to the dict	2021-05-19 17:00:40 +05:30
pukkandan	4ec82a72bb	Ensure `post_extract` and `pre_process` only run once Previously, they ran once for each format requested	2021-05-19 16:48:22 +05:30
pukkandan	07cce701de	[cleanup] linter, code formatting and readme	2021-05-19 16:48:20 +05:30
king-millezandGitHub	74e001af1d	[tenplay] Fix extractor (#314 ) Authored by: king-millez	2021-05-19 16:43:34 +05:30
pukkandan	ff2751ac9c	[youtube] Always extract `maxresdefault` thumbnail Fixes: https://github.com/ytdl-org/youtube-dl/issues/29049	2021-05-18 19:31:17 +05:30
pukkandan	abcdd12b26	[youtube:tab] Support youtube music `MP` pages	2021-05-18 19:31:08 +05:30
pukkandan	18db754858	[youtube:tab] Redirect `UC` channels that doesn't have a `videos` tab Many topic URLs don't have a videos tab, but has an equivalent `UU` playlist. If there is no playlist, fallback to using channel page	2021-05-18 19:31:07 +05:30
pukkandan	fe03a6cdc8	[youtube:tab] Support youtube music `VL` and `browse` pages	2021-05-18 19:31:06 +05:30
pukkandan	cd684175ad	[youtube:tab] Support channel search Fixes: https://github.com/ytdl-org/youtube-dl/issues/29071	2021-05-18 19:30:21 +05:30
pukkandan	da692b7920	[cleanup] youtube tests	2021-05-18 18:10:15 +05:30
pukkandan	95c01b6c16	[youtube:tab] Show alerts only from the final webpage	2021-05-18 18:09:04 +05:30
pukkandan	6911e11edd	[test:download] Only extract enough videos for `playlist_mincount`	2021-05-18 18:08:55 +05:30
pukkandan	5112f26a60	Add `pl_thumbnail` outtmpl key for playlist thumbnails This should have been implemented in `681de68e9d`, but I forgot	2021-05-18 17:12:20 +05:30
pukkandan	a06916d98e	[extractor] Add `write_debug` and `get_param`	2021-05-17 18:59:51 +05:30
pukkandan	681de68e9d	Write thumbnail of playlist Related: https://github.com/ytdl-org/youtube-dl/pull/28872, https://github.com/ytdl-org/youtube-dl/pull/28860 This is slightly different from the above PRs in that this downloads the playlist's thumbnail instead of the uploader's profile picture. But for youtube channel URLs these are the same	2021-05-17 18:24:17 +05:30
pukkandan	7aee40c13c	Fix bug in listing subtitles Bug introduced by: `2412044c90`	2021-05-17 18:24:16 +05:30
coletdjnz GitHubcraftingmod, colethedj, pukkandan	9297939ec3	[Youtube] Extract more formats for `music.youtube` URLs (#311 ) Based on: https://github.com/ytdl-org/youtube-dl/pull/28778, https://github.com/ytdl-org/youtube-dl/pull/26160 Co-authored-by: craftingmod, colethedj, pukkandan	2021-05-15 20:08:47 +05:30
pukkandan	774d79cc4c	[youtube] Add language names Co-authored by: nixxo, tpikonen Based on: https://github.com/ytdl-org/youtube-dl/pull/26112 Closes: #310	2021-05-15 19:27:53 +05:30
pukkandan	2412044c90	Add field `name` for subtitles Co-authored by: pukkandan, tpikonen Based on: #310, https://github.com/ytdl-org/youtube-dl/pull/26112	2021-05-15 19:27:52 +05:30
pukkandan	120916dac2	[youtube] multiple subtitles in same language Fixes: https://github.com/ytdl-org/youtube-dl/issues/21164 Related: #310, https://github.com/ytdl-org/youtube-dl/pull/26112	2021-05-15 19:27:48 +05:30
pukkandan	fe346461ff	Fix `--check-formats` when there is network error	2021-05-15 19:26:01 +05:30
pukkandan	d2a1fad968	[compat] Fix py2	2021-05-14 13:35:13 +05:30
pukkandan	0fb983f62d	[youtube] Extract audio language	2021-05-14 13:15:48 +05:30
pukkandan	53c18592d3	Add option `--print` Deprecates: `--get-description`, `--get-duration`, `--get-filename`, `--get-format`, `--get-id`, `--get-thumbnail`, `--get-title`, `--get-url` Closes #295	2021-05-14 13:15:47 +05:30
pukkandan	e632bce2e4	[options] Refactor callbacks	2021-05-14 13:15:47 +05:30
pukkandan	0760b0a7e2	Standardize `write_debug`	2021-05-14 13:15:29 +05:30
pukkandan	d908aa636a	[cleanup] Fix typos	2021-05-11 23:34:40 +05:30
pukkandan	3d89341b47	[common] bugfix for when `compat_opts` is not given	2021-05-11 23:29:26 +05:30
pukkandan	d8ec40b39f	[rmcdecouverte] Generalize `_VALID_URL` Closes #291	2021-05-11 18:57:55 +05:30
pukkandan	4171221823	Add compat-option `no-attach-infojson`	2021-05-11 14:25:31 +05:30
pukkandan	eaeca38fc4	[version] update :ci skip all	2021-05-11 13:42:58 +05:30