mirror of
https://github.com/yt-dlp/yt-dlp.git
synced 2026-08-10 22:28:37 +03:00
Compare commits
43
Commits
2021.05.11
...
2021.05.20
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
65075de35b | ||
|
|
50755db160 | ||
|
|
5014558ab9 | ||
|
|
28b0eb0f65 | ||
|
|
95131b2176 | ||
|
|
2305e2e5c9 | ||
|
|
00ae27690d | ||
|
|
9d5d4d64f8 | ||
|
|
98784ef8d6 | ||
|
|
d3fc8074a4 | ||
|
|
9c2b75b561 | ||
|
|
856bb8f99d | ||
|
|
af32f40bf5 | ||
|
|
4ec82a72bb | ||
|
|
07cce701de | ||
|
|
74e001af1d | ||
|
|
ff2751ac9c | ||
|
|
abcdd12b26 | ||
|
|
18db754858 | ||
|
|
fe03a6cdc8 | ||
|
|
cd684175ad | ||
|
|
da692b7920 | ||
|
|
95c01b6c16 | ||
|
|
6911e11edd | ||
|
|
5112f26a60 | ||
|
|
a06916d98e | ||
|
|
681de68e9d | ||
|
|
7aee40c13c | ||
|
|
9297939ec3 | ||
|
|
774d79cc4c | ||
|
|
2412044c90 | ||
|
|
120916dac2 | ||
|
|
fe346461ff | ||
|
|
d2a1fad968 | ||
|
|
0fb983f62d | ||
|
|
53c18592d3 | ||
|
|
e632bce2e4 | ||
|
|
0760b0a7e2 | ||
|
|
d908aa636a | ||
|
|
3d89341b47 | ||
|
|
d8ec40b39f | ||
|
|
4171221823 | ||
|
|
eaeca38fc4 |
@@ -21,7 +21,7 @@ assignees: ''
|
|||||||
|
|
||||||
<!--
|
<!--
|
||||||
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
|
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
|
||||||
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.04.22. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
|
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.05.11. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
|
||||||
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
|
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
|
||||||
- Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in https://github.com/yt-dlp/yt-dlp.
|
- Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in https://github.com/yt-dlp/yt-dlp.
|
||||||
- Search the bugtracker for similar issues: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
|
- Search the bugtracker for similar issues: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
|
||||||
@@ -29,7 +29,7 @@ Carefully read and work through this check list in order to prevent the most com
|
|||||||
-->
|
-->
|
||||||
|
|
||||||
- [ ] I'm reporting a broken site support
|
- [ ] I'm reporting a broken site support
|
||||||
- [ ] I've verified that I'm running yt-dlp version **2021.04.22**
|
- [ ] I've verified that I'm running yt-dlp version **2021.05.11**
|
||||||
- [ ] I've checked that all provided URLs are alive and playable in a browser
|
- [ ] I've checked that all provided URLs are alive and playable in a browser
|
||||||
- [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
|
- [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
|
||||||
- [ ] I've searched the bugtracker for similar issues including closed ones
|
- [ ] I've searched the bugtracker for similar issues including closed ones
|
||||||
@@ -44,7 +44,7 @@ Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your com
|
|||||||
[debug] User config: []
|
[debug] User config: []
|
||||||
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
|
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
|
||||||
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
|
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
|
||||||
[debug] yt-dlp version 2021.04.22
|
[debug] yt-dlp version 2021.05.11
|
||||||
[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
|
[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
|
||||||
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
|
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
|
||||||
[debug] Proxy map: {}
|
[debug] Proxy map: {}
|
||||||
|
|||||||
@@ -21,7 +21,7 @@ assignees: ''
|
|||||||
|
|
||||||
<!--
|
<!--
|
||||||
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
|
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
|
||||||
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.04.22. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
|
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.05.11. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
|
||||||
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
|
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
|
||||||
- Make sure that site you are requesting is not dedicated to copyright infringement, see https://github.com/yt-dlp/yt-dlp. yt-dlp does not support such sites. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
|
- Make sure that site you are requesting is not dedicated to copyright infringement, see https://github.com/yt-dlp/yt-dlp. yt-dlp does not support such sites. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
|
||||||
- Search the bugtracker for similar site support requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
|
- Search the bugtracker for similar site support requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
|
||||||
@@ -29,7 +29,7 @@ Carefully read and work through this check list in order to prevent the most com
|
|||||||
-->
|
-->
|
||||||
|
|
||||||
- [ ] I'm reporting a new site support request
|
- [ ] I'm reporting a new site support request
|
||||||
- [ ] I've verified that I'm running yt-dlp version **2021.04.22**
|
- [ ] I've verified that I'm running yt-dlp version **2021.05.11**
|
||||||
- [ ] I've checked that all provided URLs are alive and playable in a browser
|
- [ ] I've checked that all provided URLs are alive and playable in a browser
|
||||||
- [ ] I've checked that none of provided URLs violate any copyrights
|
- [ ] I've checked that none of provided URLs violate any copyrights
|
||||||
- [ ] I've searched the bugtracker for similar site support requests including closed ones
|
- [ ] I've searched the bugtracker for similar site support requests including closed ones
|
||||||
|
|||||||
@@ -21,13 +21,13 @@ assignees: ''
|
|||||||
|
|
||||||
<!--
|
<!--
|
||||||
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
|
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
|
||||||
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.04.22. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
|
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.05.11. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
|
||||||
- Search the bugtracker for similar site feature requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
|
- Search the bugtracker for similar site feature requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
|
||||||
- Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
|
- Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
|
||||||
-->
|
-->
|
||||||
|
|
||||||
- [ ] I'm reporting a site feature request
|
- [ ] I'm reporting a site feature request
|
||||||
- [ ] I've verified that I'm running yt-dlp version **2021.04.22**
|
- [ ] I've verified that I'm running yt-dlp version **2021.05.11**
|
||||||
- [ ] I've searched the bugtracker for similar site feature requests including closed ones
|
- [ ] I've searched the bugtracker for similar site feature requests including closed ones
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -21,7 +21,7 @@ assignees: ''
|
|||||||
|
|
||||||
<!--
|
<!--
|
||||||
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
|
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
|
||||||
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.04.22. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
|
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.05.11. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
|
||||||
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
|
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
|
||||||
- Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in https://github.com/yt-dlp/yt-dlp.
|
- Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in https://github.com/yt-dlp/yt-dlp.
|
||||||
- Search the bugtracker for similar issues: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
|
- Search the bugtracker for similar issues: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
|
||||||
@@ -30,7 +30,7 @@ Carefully read and work through this check list in order to prevent the most com
|
|||||||
-->
|
-->
|
||||||
|
|
||||||
- [ ] I'm reporting a broken site support issue
|
- [ ] I'm reporting a broken site support issue
|
||||||
- [ ] I've verified that I'm running yt-dlp version **2021.04.22**
|
- [ ] I've verified that I'm running yt-dlp version **2021.05.11**
|
||||||
- [ ] I've checked that all provided URLs are alive and playable in a browser
|
- [ ] I've checked that all provided URLs are alive and playable in a browser
|
||||||
- [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
|
- [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
|
||||||
- [ ] I've searched the bugtracker for similar bug reports including closed ones
|
- [ ] I've searched the bugtracker for similar bug reports including closed ones
|
||||||
@@ -46,7 +46,7 @@ Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your com
|
|||||||
[debug] User config: []
|
[debug] User config: []
|
||||||
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
|
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
|
||||||
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
|
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
|
||||||
[debug] yt-dlp version 2021.04.22
|
[debug] yt-dlp version 2021.05.11
|
||||||
[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
|
[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
|
||||||
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
|
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
|
||||||
[debug] Proxy map: {}
|
[debug] Proxy map: {}
|
||||||
|
|||||||
@@ -21,13 +21,13 @@ assignees: ''
|
|||||||
|
|
||||||
<!--
|
<!--
|
||||||
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
|
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
|
||||||
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.04.22. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
|
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.05.11. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
|
||||||
- Search the bugtracker for similar feature requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
|
- Search the bugtracker for similar feature requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
|
||||||
- Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
|
- Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
|
||||||
-->
|
-->
|
||||||
|
|
||||||
- [ ] I'm reporting a feature request
|
- [ ] I'm reporting a feature request
|
||||||
- [ ] I've verified that I'm running yt-dlp version **2021.04.22**
|
- [ ] I've verified that I'm running yt-dlp version **2021.05.11**
|
||||||
- [ ] I've searched the bugtracker for similar feature requests including closed ones
|
- [ ] I've searched the bugtracker for similar feature requests including closed ones
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -44,3 +44,7 @@ Lamieur
|
|||||||
tsukumijima
|
tsukumijima
|
||||||
Hadi0609
|
Hadi0609
|
||||||
b5eff52
|
b5eff52
|
||||||
|
craftingmod
|
||||||
|
tpikonen
|
||||||
|
tripulse
|
||||||
|
king-millez
|
||||||
|
|||||||
+39
-4
@@ -19,16 +19,51 @@
|
|||||||
-->
|
-->
|
||||||
|
|
||||||
|
|
||||||
### 2021.05.11
|
### 2021.05.20
|
||||||
|
|
||||||
|
**Youtube improvements**:
|
||||||
|
* Support youtube music `MP`, `VL` and `browse` pages
|
||||||
|
* Extract more formats for youtube music by @craftingmod, @colethedj, @pukkandan
|
||||||
|
* Extract multiple subtitles in same language by @pukkandan and @tpikonen
|
||||||
|
* Redirect channels that doesn't have a `videos` tab to their `UU` playlists
|
||||||
|
* Support in-channel search
|
||||||
|
* Sort audio-only formats correctly
|
||||||
|
* Always extract `maxresdefault` thumbnail
|
||||||
|
* Extract audio language
|
||||||
|
* Add subtitle language names by @nixxo and @tpikonen
|
||||||
|
* Show alerts only from the final webpage
|
||||||
|
* Add `html5=1` param to `get_video_info` page requests by @colethedj
|
||||||
|
* Better message when login required
|
||||||
|
**Add option `--print`**: to print any field/template
|
||||||
|
Deprecates: `--get-description`, `--get-duration`, `--get-filename`, `--get-format`, `--get-id`, `--get-thumbnail`, `--get-title`, `--get-url`
|
||||||
|
* Field `additional_urls` to download additional videos from metadata using [`--parse-metadata`](https://github.com/yt-dlp/yt-dlp#modifying-metadata)
|
||||||
|
* Merge youtube-dl: Upto [commit/dfbbe29](https://github.com/ytdl-org/youtube-dl/commit/dfbbe2902fc67f0f93ee47a8077c148055c67a9b)
|
||||||
|
* Write thumbnail of playlist and add `pl_thumbnail` outtmpl key
|
||||||
|
* [embedthumbnail] Add `flac` support and refactor `mutagen` code by @pukkandan and @tripulse
|
||||||
|
* [audius:artist] Add extractor by @king-millez
|
||||||
|
* [parlview] Add extractor by @king-millez
|
||||||
|
* [tenplay] Fix extractor by @king-millez
|
||||||
|
* [rmcdecouverte] Generalize `_VALID_URL`
|
||||||
|
* Add compat-option `no-attach-infojson`
|
||||||
|
* Add field `name` for subtitles
|
||||||
|
* Ensure `post_extract` and `pre_process` only run once
|
||||||
|
* Fix `--check-formats` when there is network error
|
||||||
|
* Standardize `write_debug` and `get_param`
|
||||||
|
* [options] Alias `--write-comments`, `--no-write-comments`
|
||||||
|
* [options] Refactor callbacks
|
||||||
|
* [test:download] Only extract enough videos for `playlist_mincount`
|
||||||
|
* [extractor] bugfix for when `compat_opts` is not given
|
||||||
|
* [cleanup] code formatting, youtube tests and readme
|
||||||
|
|
||||||
|
### 2021.05.11
|
||||||
* **Deprecate support for python versions < 3.6**
|
* **Deprecate support for python versions < 3.6**
|
||||||
* **Subtitle extraction from manifests** by [fstirlitz](https://github.com/fstirlitz). See [be6202f12b97858b9d716e608394b51065d0419f](https://github.com/yt-dlp/yt-dlp/commit/be6202f12b97858b9d716e608394b51065d0419f) for details
|
* **Subtitle extraction from manifests** by [fstirlitz](https://github.com/fstirlitz). See [be6202f](https://github.com/yt-dlp/yt-dlp/commit/be6202f12b97858b9d716e608394b51065d0419f) for details
|
||||||
* **Improve output template:**
|
* **Improve output template:**
|
||||||
* Allow slicing lists/strings using `field.start:end:step`
|
* Allow slicing lists/strings using `field.start:end:step`
|
||||||
* A field can also be used as offset like `field1+num+field2`
|
* A field can also be used as offset like `field1+num+field2`
|
||||||
* A default value can be given using `field|default`
|
* A default value can be given using `field|default`
|
||||||
* Prevent invalid fields from causing errors
|
* Prevent invalid fields from causing errors
|
||||||
* Merge youtube-dl: Upto [commit/a726009](https://github.com/ytdl-org/youtube-dl/commit/a7260099873acc6dc7d76cafad2f6b139087afd0)
|
* **Merge youtube-dl**: Upto [commit/a726009](https://github.com/ytdl-org/youtube-dl/commit/a7260099873acc6dc7d76cafad2f6b139087afd0)
|
||||||
* **Remove options** `-l`, `-t`, `-A` completely and disable `--auto-number`, `--title`, `--literal`, `--id`
|
* **Remove options** `-l`, `-t`, `-A` completely and disable `--auto-number`, `--title`, `--literal`, `--id`
|
||||||
* [Plugins] Prioritize plugins over standard extractors and prevent plugins from overwriting the standard extractor classes
|
* [Plugins] Prioritize plugins over standard extractors and prevent plugins from overwriting the standard extractor classes
|
||||||
* [downloader] Fix `quiet` and `to_stderr`
|
* [downloader] Fix `quiet` and `to_stderr`
|
||||||
@@ -63,7 +98,7 @@
|
|||||||
* [zee5] Fix py2 compatibility
|
* [zee5] Fix py2 compatibility
|
||||||
* Fix `playlist_index` and add `playlist_autonumber`. See [#302](https://github.com/yt-dlp/yt-dlp/issues/302) for details
|
* Fix `playlist_index` and add `playlist_autonumber`. See [#302](https://github.com/yt-dlp/yt-dlp/issues/302) for details
|
||||||
* Add experimental option `--check-formats` to test the URLs before format selection
|
* Add experimental option `--check-formats` to test the URLs before format selection
|
||||||
* Option `--compat-options` to revert some of yt-dlp's changes
|
* Option `--compat-options` to revert [some of yt-dlp's changes](https://github.com/yt-dlp/yt-dlp#differences-in-default-behavior)
|
||||||
* Deprecates `--list-formats-as-table`, `--list-formats-old`
|
* Deprecates `--list-formats-as-table`, `--list-formats-old`
|
||||||
* Fix number of digits in `%(playlist_index)s`
|
* Fix number of digits in `%(playlist_index)s`
|
||||||
* Fix case sensitivity of format selector
|
* Fix case sensitivity of format selector
|
||||||
|
|||||||
@@ -66,15 +66,17 @@ The major new features from the latest release of [blackjack4494/yt-dlc](https:/
|
|||||||
|
|
||||||
* **[Format Sorting](#sorting-formats)**: The default format sorting options have been changed so that higher resolution and better codecs will be now preferred instead of simply using larger bitrate. Furthermore, you can now specify the sort order using `-S`. This allows for much easier format selection that what is possible by simply using `--format` ([examples](#format-selection-examples))
|
* **[Format Sorting](#sorting-formats)**: The default format sorting options have been changed so that higher resolution and better codecs will be now preferred instead of simply using larger bitrate. Furthermore, you can now specify the sort order using `-S`. This allows for much easier format selection that what is possible by simply using `--format` ([examples](#format-selection-examples))
|
||||||
|
|
||||||
* **Merged with youtube-dl [commit/a726009](https://github.com/ytdl-org/youtube-dl/commit/a7260099873acc6dc7d76cafad2f6b139087afd0)**: (v2021.04.26) You get all the latest features and patches of [youtube-dl](https://github.com/ytdl-org/youtube-dl) in addition to all the features of [youtube-dlc](https://github.com/blackjack4494/yt-dlc)
|
* **Merged with youtube-dl [commit/dfbbe29](https://github.com/ytdl-org/youtube-dl/commit/dfbbe2902fc67f0f93ee47a8077c148055c67a9b)**: (v2021.05.16) You get all the latest features and patches of [youtube-dl](https://github.com/ytdl-org/youtube-dl) in addition to all the features of [youtube-dlc](https://github.com/blackjack4494/yt-dlc)
|
||||||
|
|
||||||
* **Merged with animelover1984/youtube-dl**: You get most of the features and improvements from [animelover1984/youtube-dl](https://github.com/animelover1984/youtube-dl) including `--get-comments`, `BiliBiliSearch`, `BilibiliChannel`, Embedding thumbnail in mp4/ogg/opus, playlist infojson etc. Note that the NicoNico improvements are not available. See [#31](https://github.com/yt-dlp/yt-dlp/pull/31) for details.
|
* **Merged with animelover1984/youtube-dl**: You get most of the features and improvements from [animelover1984/youtube-dl](https://github.com/animelover1984/youtube-dl) including `--write-comments`, `BiliBiliSearch`, `BilibiliChannel`, Embedding thumbnail in mp4/ogg/opus, playlist infojson etc. Note that the NicoNico improvements are not available. See [#31](https://github.com/yt-dlp/yt-dlp/pull/31) for details.
|
||||||
|
|
||||||
* **Youtube improvements**:
|
* **Youtube improvements**:
|
||||||
* All Youtube Feeds (`:ytfav`, `:ytwatchlater`, `:ytsubs`, `:ythistory`, `:ytrec`) works and supports downloading multiple pages of content
|
* All Feeds (`:ytfav`, `:ytwatchlater`, `:ytsubs`, `:ythistory`, `:ytrec`) supports downloading multiple pages of content
|
||||||
* Youtube search (`ytsearch:`, `ytsearchdate:`) along with Search URLs work
|
* Search (`ytsearch:`, `ytsearchdate:`), search URLs and in-channel search works
|
||||||
* Youtube mixes supports downloading multiple pages of content
|
* Mixes supports downloading multiple pages of content
|
||||||
* Redirect channel's home URL automatically to `/video` to preserve the old behaviour
|
* Redirect channel's home URL automatically to `/video` to preserve the old behaviour
|
||||||
|
* `255kbps` audio is extracted from youtube music if premium cookies are given
|
||||||
|
* Youtube music Albums, channels etc can be downloaded
|
||||||
|
|
||||||
* **Split video by chapters**: Videos can be split into multiple files based on chapters using `--split-chapters`
|
* **Split video by chapters**: Videos can be split into multiple files based on chapters using `--split-chapters`
|
||||||
|
|
||||||
@@ -82,9 +84,9 @@ The major new features from the latest release of [blackjack4494/yt-dlc](https:/
|
|||||||
|
|
||||||
* **Aria2c with HLS/DASH**: You can use `aria2c` as the external downloader for DASH(mpd) and HLS(m3u8) formats
|
* **Aria2c with HLS/DASH**: You can use `aria2c` as the external downloader for DASH(mpd) and HLS(m3u8) formats
|
||||||
|
|
||||||
* **New extractors**: AnimeLab, Philo MSO, Rcs, Gedi, bitwave.tv, mildom, audius, zee5, mtv.it, wimtv, pluto.tv, niconico users, discoveryplus.in, mediathek, NFHSNetwork, nebula, ukcolumn, whowatch, MxplayerShow
|
* **New extractors**: AnimeLab, Philo MSO, Rcs, Gedi, bitwave.tv, mildom, audius, zee5, mtv.it, wimtv, pluto.tv, niconico users, discoveryplus.in, mediathek, NFHSNetwork, nebula, ukcolumn, whowatch, MxplayerShow, parlview (au)
|
||||||
|
|
||||||
* **Fixed extractors**: archive.org, roosterteeth.com, skyit, instagram, itv, SouthparkDe, spreaker, Vlive, akamai, ina, rumble, tennistv, amcnetworks, la7 podcasts, linuxacadamy, nitter, twitcasting, viu, crackle, curiositystream, mediasite, rmcdecouverte, sonyliv, tubi
|
* **Fixed extractors**: archive.org, roosterteeth.com, skyit, instagram, itv, SouthparkDe, spreaker, Vlive, akamai, ina, rumble, tennistv, amcnetworks, la7 podcasts, linuxacadamy, nitter, twitcasting, viu, crackle, curiositystream, mediasite, rmcdecouverte, sonyliv, tubi, tenplay
|
||||||
|
|
||||||
* **Subtitle extraction from manifests**: Subtitles can be extracted from streaming media manifests. See [be6202f12b97858b9d716e608394b51065d0419f](https://github.com/yt-dlp/yt-dlp/commit/be6202f12b97858b9d716e608394b51065d0419f) for details
|
* **Subtitle extraction from manifests**: Subtitles can be extracted from streaming media manifests. See [be6202f12b97858b9d716e608394b51065d0419f](https://github.com/yt-dlp/yt-dlp/commit/be6202f12b97858b9d716e608394b51065d0419f) for details
|
||||||
|
|
||||||
@@ -114,24 +116,25 @@ If you are coming from [youtube-dl](https://github.com/ytdl-org/youtube-dl), the
|
|||||||
|
|
||||||
Some of yt-dlp's default options are different from that of youtube-dl and youtube-dlc.
|
Some of yt-dlp's default options are different from that of youtube-dl and youtube-dlc.
|
||||||
|
|
||||||
1. The options `--id`, `--auto-number` (`-A`), `--title` (`-t`) and `--literal` (`-l`), no longer work. See [removed options](#Removed) for details
|
* The options `--id`, `--auto-number` (`-A`), `--title` (`-t`) and `--literal` (`-l`), no longer work. See [removed options](#Removed) for details
|
||||||
1. `avconv` is not supported as as an alternative to `ffmpeg`
|
* `avconv` is not supported as as an alternative to `ffmpeg`
|
||||||
1. The default [output template](#output-template) is `%(title)s [%(id)s].%(ext)s`. There is no real reason for this change. This was changed before yt-dlp was ever made public and now there are no plans to change it back to `%(title)s.%(id)s.%(ext)s`. Instead, you may use `--compat-options filename`
|
* The default [output template](#output-template) is `%(title)s [%(id)s].%(ext)s`. There is no real reason for this change. This was changed before yt-dlp was ever made public and now there are no plans to change it back to `%(title)s.%(id)s.%(ext)s`. Instead, you may use `--compat-options filename`
|
||||||
1. The default [format sorting](sorting-formats) is different from youtube-dl and prefers higher resolution and better codecs rather than higher bitrates. You can use the `--format-sort` option to change this to any order you prefer, or use `--compat-options format-sort` to use youtube-dl's sorting order
|
* The default [format sorting](sorting-formats) is different from youtube-dl and prefers higher resolution and better codecs rather than higher bitrates. You can use the `--format-sort` option to change this to any order you prefer, or use `--compat-options format-sort` to use youtube-dl's sorting order
|
||||||
1. The default format selector is `bv*+ba/b`. This means that if a combined video + audio format that is better than the best video-only format is found, the former will be prefered. Use `-f bv+ba/b` or `--compat-options format-spec` to revert this
|
* The default format selector is `bv*+ba/b`. This means that if a combined video + audio format that is better than the best video-only format is found, the former will be prefered. Use `-f bv+ba/b` or `--compat-options format-spec` to revert this
|
||||||
1. Unlike youtube-dlc, yt-dlp does not allow merging multiple audio/video streams into one file by default (since this conflicts with the use of `-f bv*+ba`). If needed, this feature must be enabled using `--audio-multistreams` and `--video-multistreams`. You can also use `--compat-options multistreams` to enable both
|
* Unlike youtube-dlc, yt-dlp does not allow merging multiple audio/video streams into one file by default (since this conflicts with the use of `-f bv*+ba`). If needed, this feature must be enabled using `--audio-multistreams` and `--video-multistreams`. You can also use `--compat-options multistreams` to enable both
|
||||||
1. `--ignore-errors` is enabled by default. Use `--abort-on-error` or `--compat-options abort-on-error` to abort on errors instead
|
* `--ignore-errors` is enabled by default. Use `--abort-on-error` or `--compat-options abort-on-error` to abort on errors instead
|
||||||
1. When writing metadata files such as thumbnails, description or infojson, the same information (if available) is also written for playlists. Use `--no-write-playlist-metafiles` or `--compat-options no-playlist-metafiles` to not write these files
|
* When writing metadata files such as thumbnails, description or infojson, the same information (if available) is also written for playlists. Use `--no-write-playlist-metafiles` or `--compat-options no-playlist-metafiles` to not write these files
|
||||||
1. `playlist_index` behaves differently when used with options like `--playlist-reverse` and `--playlist-items`. See [#302](https://github.com/yt-dlp/yt-dlp/issues/302) for details. You can use `--compat-options playlist-index` if you want to keep the earlier behavior
|
* `--add-metadata` attaches the `infojson` to `mkv` files in addition to writing the metadata when used with `--write-infojson`. Use `--compat-options no-attach-info-json` to revert this
|
||||||
1. The output of `-F` is listed in a new format. Use `--compat-options list-formats` to revert this
|
* `playlist_index` behaves differently when used with options like `--playlist-reverse` and `--playlist-items`. See [#302](https://github.com/yt-dlp/yt-dlp/issues/302) for details. You can use `--compat-options playlist-index` if you want to keep the earlier behavior
|
||||||
1. Youtube live chat (if available) is considered as a subtitle. Use `--sub-langs all,-live_chat` to download all subtitles except live chat. You can also use `--compat-options no-live-chat` to prevent live chat from downloading
|
* The output of `-F` is listed in a new format. Use `--compat-options list-formats` to revert this
|
||||||
1. Youtube channel URLs are automatically redirected to `/video`. Either append a `/featured` to the URL or use `--compat-options no-youtube-channel-redirect` to download only the videos in the home page
|
* Youtube live chat (if available) is considered as a subtitle. Use `--sub-langs all,-live_chat` to download all subtitles except live chat. You can also use `--compat-options no-live-chat` to prevent live chat from downloading
|
||||||
1. Unavailable videos are also listed for youtube playlists. Use `--compat-options no-youtube-unavailable-videos` to remove this
|
* Youtube channel URLs are automatically redirected to `/video`. Either append a `/featured` to the URL or use `--compat-options no-youtube-channel-redirect` to download only the videos in the home page
|
||||||
|
* Unavailable videos are also listed for youtube playlists. Use `--compat-options no-youtube-unavailable-videos` to remove this
|
||||||
|
|
||||||
For ease of use, a few more compat options are available:
|
For ease of use, a few more compat options are available:
|
||||||
1. `--compat-options all` = Use all compat options
|
* `--compat-options all`: Use all compat options
|
||||||
1. `--compat-options youtube-dl` = `--compat-options all,-multistreams`
|
* `--compat-options youtube-dl`: Same as `--compat-options all,-multistreams`
|
||||||
1. `--compat-options youtube-dlc` = `--compat-options all,-no-live-chat,-no-youtube-channel-redirect`
|
* `--compat-options youtube-dlc`: Same as `--compat-options all,-no-live-chat,-no-youtube-channel-redirect`
|
||||||
|
|
||||||
|
|
||||||
# INSTALLATION
|
# INSTALLATION
|
||||||
@@ -167,7 +170,7 @@ Python versions 3.6+ (CPython and PyPy) are officially supported. Other versions
|
|||||||
|
|
||||||
On windows, [Microsoft Visual C++ 2010 Redistributable Package (x86)](https://www.microsoft.com/en-us/download/details.aspx?id=26999) is also necessary to run yt-dlp. You probably already have this, but if the executable throws an error due to missing `MSVCR100.dll` you need to install it.
|
On windows, [Microsoft Visual C++ 2010 Redistributable Package (x86)](https://www.microsoft.com/en-us/download/details.aspx?id=26999) is also necessary to run yt-dlp. You probably already have this, but if the executable throws an error due to missing `MSVCR100.dll` you need to install it.
|
||||||
|
|
||||||
Although there are no other required dependencies, `ffmpeg` and `ffprobe` are highly recommended. Other optional dependencies are `sponskrub`, `AtomicParsley`, `mutagen`, `pycryptodome` and any of the supported external downloaders. Note that the windows releases are already built with the python interpreter, mutagen and pycryptodome included.
|
Although there are no other required dependencies, `ffmpeg` and `ffprobe` are highly recommended. Other optional dependencies are `sponskrub`, `AtomicParsley`, `mutagen`, `pycryptodome`, `phantomjs` and any of the supported external downloaders. Note that the windows releases are already built with the python interpreter, mutagen and pycryptodome included.
|
||||||
|
|
||||||
### UPDATE
|
### UPDATE
|
||||||
You can use `yt-dlp -U` to update if you are using the provided release.
|
You can use `yt-dlp -U` to update if you are using the provided release.
|
||||||
@@ -482,10 +485,13 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
|
|||||||
could still contain some personal
|
could still contain some personal
|
||||||
information (default)
|
information (default)
|
||||||
--no-clean-infojson Write all fields to the infojson
|
--no-clean-infojson Write all fields to the infojson
|
||||||
--get-comments Retrieve video comments to be placed in the
|
--write-comments Retrieve video comments to be placed in the
|
||||||
.info.json file. The comments are fetched
|
infojson. The comments are fetched even
|
||||||
even without this option if the extraction
|
without this option if the extraction is
|
||||||
is known to be quick
|
known to be quick (Alias: --get-comments)
|
||||||
|
--no-write-comments Do not retrieve video comments unless the
|
||||||
|
extraction is known to be quick
|
||||||
|
(Alias: --no-get-comments)
|
||||||
--load-info-json FILE JSON file containing the video information
|
--load-info-json FILE JSON file containing the video information
|
||||||
(created with the "--write-info-json"
|
(created with the "--write-info-json"
|
||||||
option)
|
option)
|
||||||
@@ -533,14 +539,9 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
|
|||||||
formats are found (default)
|
formats are found (default)
|
||||||
--skip-download Do not download the video but write all
|
--skip-download Do not download the video but write all
|
||||||
related files (Alias: --no-download)
|
related files (Alias: --no-download)
|
||||||
-g, --get-url Simulate, quiet but print URL
|
-O, --print TEMPLATE Simulate, quiet but print the given fields.
|
||||||
-e, --get-title Simulate, quiet but print title
|
Either a field name or similar formatting
|
||||||
--get-id Simulate, quiet but print id
|
as the output template can be used
|
||||||
--get-thumbnail Simulate, quiet but print thumbnail URL
|
|
||||||
--get-description Simulate, quiet but print video description
|
|
||||||
--get-duration Simulate, quiet but print video length
|
|
||||||
--get-filename Simulate, quiet but print output filename
|
|
||||||
--get-format Simulate, quiet but print output format
|
|
||||||
-j, --dump-json Simulate, quiet but print JSON information.
|
-j, --dump-json Simulate, quiet but print JSON information.
|
||||||
See "OUTPUT TEMPLATE" for a description of
|
See "OUTPUT TEMPLATE" for a description of
|
||||||
available keys
|
available keys
|
||||||
@@ -822,7 +823,7 @@ You can configure yt-dlp by placing any supported command line option to a confi
|
|||||||
* `~/yt-dlp.conf.txt`
|
* `~/yt-dlp.conf.txt`
|
||||||
|
|
||||||
Note that `~` points to `C:\Users\<user name>` on windows. Also, `%XDG_CONFIG_HOME%` defaults to `~/.config` if undefined
|
Note that `~` points to `C:\Users\<user name>` on windows. Also, `%XDG_CONFIG_HOME%` defaults to `~/.config` if undefined
|
||||||
1. **System Configuration**: `/etc/yt-dlp.conf` or `/etc/yt-dlp.conf`
|
1. **System Configuration**: `/etc/yt-dlp.conf`
|
||||||
|
|
||||||
For example, with the following configuration file yt-dlp will always extract the audio, not copy the mtime, use a proxy and save all videos under `YouTube` directory in your home directory:
|
For example, with the following configuration file yt-dlp will always extract the audio, not copy the mtime, use a proxy and save all videos under `YouTube` directory in your home directory:
|
||||||
```
|
```
|
||||||
@@ -889,7 +890,7 @@ To summarize, the general syntax for a field is:
|
|||||||
%(name[.keys][addition][>strf][|default])[flags][width][.precision][length]type
|
%(name[.keys][addition][>strf][|default])[flags][width][.precision][length]type
|
||||||
```
|
```
|
||||||
|
|
||||||
Additionally, you can set different output templates for the various metadata files separately from the general output template by specifying the type of file followed by the template separated by a colon `:`. The different file types supported are `subtitle`, `thumbnail`, `description`, `annotation`, `infojson`, `pl_description`, `pl_infojson`, `chapter`. For example, `-o '%(title)s.%(ext)s' -o 'thumbnail:%(title)s\%(title)s.%(ext)s'` will put the thumbnails in a folder with the same name as the video.
|
Additionally, you can set different output templates for the various metadata files separately from the general output template by specifying the type of file followed by the template separated by a colon `:`. The different file types supported are `subtitle`, `thumbnail`, `description`, `annotation`, `infojson`, `pl_thumbnail`, `pl_description`, `pl_infojson`, `chapter`. For example, `-o '%(title)s.%(ext)s' -o 'thumbnail:%(title)s\%(title)s.%(ext)s'` will put the thumbnails in a folder with the same name as the video.
|
||||||
|
|
||||||
The available fields are:
|
The available fields are:
|
||||||
|
|
||||||
@@ -911,7 +912,7 @@ The available fields are:
|
|||||||
- `channel_id` (string): Id of the channel
|
- `channel_id` (string): Id of the channel
|
||||||
- `location` (string): Physical location where the video was filmed
|
- `location` (string): Physical location where the video was filmed
|
||||||
- `duration` (numeric): Length of the video in seconds
|
- `duration` (numeric): Length of the video in seconds
|
||||||
- `duration_string` (string): Length of the video (HH-mm-ss)
|
- `duration_string` (string): Length of the video (HH:mm:ss)
|
||||||
- `view_count` (numeric): How many users have watched the video on the platform
|
- `view_count` (numeric): How many users have watched the video on the platform
|
||||||
- `like_count` (numeric): Number of positive ratings of the video
|
- `like_count` (numeric): Number of positive ratings of the video
|
||||||
- `dislike_count` (numeric): Number of negative ratings of the video
|
- `dislike_count` (numeric): Number of negative ratings of the video
|
||||||
@@ -989,6 +990,11 @@ Available for `chapter:` prefix when using `--split-chapters` for videos with in
|
|||||||
- `section_start` (numeric): Start time of the chapter in seconds
|
- `section_start` (numeric): Start time of the chapter in seconds
|
||||||
- `section_end` (numeric): End time of the chapter in seconds
|
- `section_end` (numeric): End time of the chapter in seconds
|
||||||
|
|
||||||
|
Available only when used in `--print`:
|
||||||
|
|
||||||
|
- `urls` (string): The URLs of all requested formats, one in each line
|
||||||
|
- `filename` (string): Name of the video file. Note that the actual filename may be different due to post-processing. Use `--exec echo` to get the name after all postprocessing is complete
|
||||||
|
|
||||||
Each aforementioned sequence when referenced in an output template will be replaced by the actual value corresponding to the sequence name. Note that some of the sequences are not guaranteed to be present since they depend on the metadata obtained by a particular extractor. Such sequences will be replaced with placeholder value provided with `--output-na-placeholder` (`NA` by default).
|
Each aforementioned sequence when referenced in an output template will be replaced by the actual value corresponding to the sequence name. Note that some of the sequences are not guaranteed to be present since they depend on the metadata obtained by a particular extractor. Such sequences will be replaced with placeholder value provided with `--output-na-placeholder` (`NA` by default).
|
||||||
|
|
||||||
For example for `-o %(title)s-%(id)s.%(ext)s` and an mp4 video with title `yt-dlp test video` and id `BaW_jenozKcj`, this will result in a `yt-dlp test video-BaW_jenozKcj.mp4` file created in the current directory.
|
For example for `-o %(title)s-%(id)s.%(ext)s` and an mp4 video with title `yt-dlp test video` and id `BaW_jenozKcj`, this will result in a `yt-dlp test video-BaW_jenozKcj.mp4` file created in the current directory.
|
||||||
@@ -1278,7 +1284,9 @@ The metadata obtained the the extractors can be modified by using `--parse-metad
|
|||||||
|
|
||||||
Note that any field created by this can be used in the [output template](#output-template) and will also affect the media file's metadata added when using `--add-metadata`.
|
Note that any field created by this can be used in the [output template](#output-template) and will also affect the media file's metadata added when using `--add-metadata`.
|
||||||
|
|
||||||
You can also use this to change only the metadata that is embedded in the media file. To do this, set the value of the corresponding field with a `meta_` prefix. For example, any value you set to `meta_description` field will be added to the `description` field in the file. You can use this to set a different "description" and "synopsis", for example.
|
This option also has a few special uses:
|
||||||
|
* You can use this to change the metadata that is embedded in the media file. To do this, set the value of the corresponding field with a `meta_` prefix. For example, any value you set to `meta_description` field will be added to the `description` field in the file. You can use this to set a different "description" and "synopsis", for example
|
||||||
|
* You can download an additional URL based on the metadata of the currently downloaded video. To do this, set the field `additional_urls` to the URL that you want to download. Eg: `--parse-metadata "description:(?P<additional_urls>https?://www\.vimeo\.com/\d+)` will download the first vimeo video found in the description
|
||||||
|
|
||||||
## Modifying metadata examples
|
## Modifying metadata examples
|
||||||
|
|
||||||
@@ -1312,6 +1320,14 @@ These are all the deprecated options and the current alternative to achieve the
|
|||||||
#### Not recommended
|
#### Not recommended
|
||||||
While these options still work, their use is not recommended since there are other alternatives to achieve the same
|
While these options still work, their use is not recommended since there are other alternatives to achieve the same
|
||||||
|
|
||||||
|
--get-description --print description
|
||||||
|
--get-duration --print duration_string
|
||||||
|
--get-filename --print filename
|
||||||
|
--get-format --print format
|
||||||
|
--get-id --print id
|
||||||
|
--get-thumbnail --print thumbnail
|
||||||
|
-e, --get-title --print title
|
||||||
|
-g, --get-url --print urls
|
||||||
--all-formats -f all
|
--all-formats -f all
|
||||||
--all-subs --sub-langs all --write-subs
|
--all-subs --sub-langs all --write-subs
|
||||||
--autonumber-size NUMBER Use string formatting. Eg: %(autonumber)03d
|
--autonumber-size NUMBER Use string formatting. Eg: %(autonumber)03d
|
||||||
@@ -1323,6 +1339,7 @@ While these options still work, their use is not recommended since there are oth
|
|||||||
--list-formats-as-table --compat-options -list-formats [Default] (Alias: --no-list-formats-old)
|
--list-formats-as-table --compat-options -list-formats [Default] (Alias: --no-list-formats-old)
|
||||||
--sponskrub-args ARGS --ppa "sponskrub:ARGS"
|
--sponskrub-args ARGS --ppa "sponskrub:ARGS"
|
||||||
--test Used by developers for testing extractors. Not intended for the end user
|
--test Used by developers for testing extractors. Not intended for the end user
|
||||||
|
--youtube-print-sig-code Used for testing youtube signatures
|
||||||
|
|
||||||
|
|
||||||
#### Old aliases
|
#### Old aliases
|
||||||
@@ -1353,7 +1370,6 @@ These options may no longer work as intended
|
|||||||
--no-call-home Default
|
--no-call-home Default
|
||||||
--include-ads No longer supported
|
--include-ads No longer supported
|
||||||
--no-include-ads Default
|
--no-include-ads Default
|
||||||
--youtube-print-sig-code No longer supported
|
|
||||||
|
|
||||||
#### Removed
|
#### Removed
|
||||||
These options were deprecated since 2014 and have now been entirely removed
|
These options were deprecated since 2014 and have now been entirely removed
|
||||||
|
|||||||
@@ -82,6 +82,7 @@
|
|||||||
- **audiomack**
|
- **audiomack**
|
||||||
- **audiomack:album**
|
- **audiomack:album**
|
||||||
- **Audius**: Audius.co
|
- **Audius**: Audius.co
|
||||||
|
- **audius:artist**: Audius.co profile/artist pages
|
||||||
- **audius:playlist**: Audius.co playlists
|
- **audius:playlist**: Audius.co playlists
|
||||||
- **audius:track**: Audius track ID or API link. Prepend with "audius:"
|
- **audius:track**: Audius track ID or API link. Prepend with "audius:"
|
||||||
- **AWAAN**
|
- **AWAAN**
|
||||||
@@ -725,6 +726,7 @@
|
|||||||
- **pandora.tv**: 판도라TV
|
- **pandora.tv**: 판도라TV
|
||||||
- **ParamountNetwork**
|
- **ParamountNetwork**
|
||||||
- **parliamentlive.tv**: UK parliament videos
|
- **parliamentlive.tv**: UK parliament videos
|
||||||
|
- **Parlview**
|
||||||
- **Patreon**
|
- **Patreon**
|
||||||
- **pbs**: Public Broadcasting Service (PBS) and member stations: PBS: Public Broadcasting Service, APT - Alabama Public Television (WBIQ), GPB/Georgia Public Broadcasting (WGTV), Mississippi Public Broadcasting (WMPN), Nashville Public Television (WNPT), WFSU-TV (WFSU), WSRE (WSRE), WTCI (WTCI), WPBA/Channel 30 (WPBA), Alaska Public Media (KAKM), Arizona PBS (KAET), KNME-TV/Channel 5 (KNME), Vegas PBS (KLVX), AETN/ARKANSAS ETV NETWORK (KETS), KET (WKLE), WKNO/Channel 10 (WKNO), LPB/LOUISIANA PUBLIC BROADCASTING (WLPB), OETA (KETA), Ozarks Public Television (KOZK), WSIU Public Broadcasting (WSIU), KEET TV (KEET), KIXE/Channel 9 (KIXE), KPBS San Diego (KPBS), KQED (KQED), KVIE Public Television (KVIE), PBS SoCal/KOCE (KOCE), ValleyPBS (KVPT), CONNECTICUT PUBLIC TELEVISION (WEDH), KNPB Channel 5 (KNPB), SOPTV (KSYS), Rocky Mountain PBS (KRMA), KENW-TV3 (KENW), KUED Channel 7 (KUED), Wyoming PBS (KCWC), Colorado Public Television / KBDI 12 (KBDI), KBYU-TV (KBYU), Thirteen/WNET New York (WNET), WGBH/Channel 2 (WGBH), WGBY (WGBY), NJTV Public Media NJ (WNJT), WLIW21 (WLIW), mpt/Maryland Public Television (WMPB), WETA Television and Radio (WETA), WHYY (WHYY), PBS 39 (WLVT), WVPT - Your Source for PBS and More! (WVPT), Howard University Television (WHUT), WEDU PBS (WEDU), WGCU Public Media (WGCU), WPBT2 (WPBT), WUCF TV (WUCF), WUFT/Channel 5 (WUFT), WXEL/Channel 42 (WXEL), WLRN/Channel 17 (WLRN), WUSF Public Broadcasting (WUSF), ETV (WRLK), UNC-TV (WUNC), PBS Hawaii - Oceanic Cable Channel 10 (KHET), Idaho Public Television (KAID), KSPS (KSPS), OPB (KOPB), KWSU/Channel 10 & KTNW/Channel 31 (KWSU), WILL-TV (WILL), Network Knowledge - WSEC/Springfield (WSEC), WTTW11 (WTTW), Iowa Public Television/IPTV (KDIN), Nine Network (KETC), PBS39 Fort Wayne (WFWA), WFYI Indianapolis (WFYI), Milwaukee Public Television (WMVS), WNIN (WNIN), WNIT Public Television (WNIT), WPT (WPNE), WVUT/Channel 22 (WVUT), WEIU/Channel 51 (WEIU), WQPT-TV (WQPT), WYCC PBS Chicago (WYCC), WIPB-TV (WIPB), WTIU (WTIU), CET (WCET), ThinkTVNetwork (WPTD), WBGU-TV (WBGU), WGVU TV (WGVU), NET1 (KUON), Pioneer Public Television (KWCM), SDPB Television (KUSD), TPT (KTCA), KSMQ (KSMQ), KPTS/Channel 8 (KPTS), KTWU/Channel 11 (KTWU), East Tennessee PBS (WSJK), WCTE-TV (WCTE), WLJT, Channel 11 (WLJT), WOSU TV (WOSU), WOUB/WOUC (WOUB), WVPB (WVPB), WKYU-PBS (WKYU), KERA 13 (KERA), MPBN (WCBB), Mountain Lake PBS (WCFE), NHPTV (WENH), Vermont PBS (WETK), witf (WITF), WQED Multimedia (WQED), WMHT Educational Telecommunications (WMHT), Q-TV (WDCQ), WTVS Detroit Public TV (WTVS), CMU Public Television (WCMU), WKAR-TV (WKAR), WNMU-TV Public TV 13 (WNMU), WDSE - WRPT (WDSE), WGTE TV (WGTE), Lakeland Public Television (KAWE), KMOS-TV - Channels 6.1, 6.2 and 6.3 (KMOS), MontanaPBS (KUSM), KRWG/Channel 22 (KRWG), KACV (KACV), KCOS/Channel 13 (KCOS), WCNY/Channel 24 (WCNY), WNED (WNED), WPBS (WPBS), WSKG Public TV (WSKG), WXXI (WXXI), WPSU (WPSU), WVIA Public Media Studios (WVIA), WTVI (WTVI), Western Reserve PBS (WNEO), WVIZ/PBS ideastream (WVIZ), KCTS 9 (KCTS), Basin PBS (KPBT), KUHT / Channel 8 (KUHT), KLRN (KLRN), KLRU (KLRU), WTJX Channel 12 (WTJX), WCVE PBS (WCVE), KBTC Public Television (KBTC)
|
- **pbs**: Public Broadcasting Service (PBS) and member stations: PBS: Public Broadcasting Service, APT - Alabama Public Television (WBIQ), GPB/Georgia Public Broadcasting (WGTV), Mississippi Public Broadcasting (WMPN), Nashville Public Television (WNPT), WFSU-TV (WFSU), WSRE (WSRE), WTCI (WTCI), WPBA/Channel 30 (WPBA), Alaska Public Media (KAKM), Arizona PBS (KAET), KNME-TV/Channel 5 (KNME), Vegas PBS (KLVX), AETN/ARKANSAS ETV NETWORK (KETS), KET (WKLE), WKNO/Channel 10 (WKNO), LPB/LOUISIANA PUBLIC BROADCASTING (WLPB), OETA (KETA), Ozarks Public Television (KOZK), WSIU Public Broadcasting (WSIU), KEET TV (KEET), KIXE/Channel 9 (KIXE), KPBS San Diego (KPBS), KQED (KQED), KVIE Public Television (KVIE), PBS SoCal/KOCE (KOCE), ValleyPBS (KVPT), CONNECTICUT PUBLIC TELEVISION (WEDH), KNPB Channel 5 (KNPB), SOPTV (KSYS), Rocky Mountain PBS (KRMA), KENW-TV3 (KENW), KUED Channel 7 (KUED), Wyoming PBS (KCWC), Colorado Public Television / KBDI 12 (KBDI), KBYU-TV (KBYU), Thirteen/WNET New York (WNET), WGBH/Channel 2 (WGBH), WGBY (WGBY), NJTV Public Media NJ (WNJT), WLIW21 (WLIW), mpt/Maryland Public Television (WMPB), WETA Television and Radio (WETA), WHYY (WHYY), PBS 39 (WLVT), WVPT - Your Source for PBS and More! (WVPT), Howard University Television (WHUT), WEDU PBS (WEDU), WGCU Public Media (WGCU), WPBT2 (WPBT), WUCF TV (WUCF), WUFT/Channel 5 (WUFT), WXEL/Channel 42 (WXEL), WLRN/Channel 17 (WLRN), WUSF Public Broadcasting (WUSF), ETV (WRLK), UNC-TV (WUNC), PBS Hawaii - Oceanic Cable Channel 10 (KHET), Idaho Public Television (KAID), KSPS (KSPS), OPB (KOPB), KWSU/Channel 10 & KTNW/Channel 31 (KWSU), WILL-TV (WILL), Network Knowledge - WSEC/Springfield (WSEC), WTTW11 (WTTW), Iowa Public Television/IPTV (KDIN), Nine Network (KETC), PBS39 Fort Wayne (WFWA), WFYI Indianapolis (WFYI), Milwaukee Public Television (WMVS), WNIN (WNIN), WNIT Public Television (WNIT), WPT (WPNE), WVUT/Channel 22 (WVUT), WEIU/Channel 51 (WEIU), WQPT-TV (WQPT), WYCC PBS Chicago (WYCC), WIPB-TV (WIPB), WTIU (WTIU), CET (WCET), ThinkTVNetwork (WPTD), WBGU-TV (WBGU), WGVU TV (WGVU), NET1 (KUON), Pioneer Public Television (KWCM), SDPB Television (KUSD), TPT (KTCA), KSMQ (KSMQ), KPTS/Channel 8 (KPTS), KTWU/Channel 11 (KTWU), East Tennessee PBS (WSJK), WCTE-TV (WCTE), WLJT, Channel 11 (WLJT), WOSU TV (WOSU), WOUB/WOUC (WOUB), WVPB (WVPB), WKYU-PBS (WKYU), KERA 13 (KERA), MPBN (WCBB), Mountain Lake PBS (WCFE), NHPTV (WENH), Vermont PBS (WETK), witf (WITF), WQED Multimedia (WQED), WMHT Educational Telecommunications (WMHT), Q-TV (WDCQ), WTVS Detroit Public TV (WTVS), CMU Public Television (WCMU), WKAR-TV (WKAR), WNMU-TV Public TV 13 (WNMU), WDSE - WRPT (WDSE), WGTE TV (WGTE), Lakeland Public Television (KAWE), KMOS-TV - Channels 6.1, 6.2 and 6.3 (KMOS), MontanaPBS (KUSM), KRWG/Channel 22 (KRWG), KACV (KACV), KCOS/Channel 13 (KCOS), WCNY/Channel 24 (WCNY), WNED (WNED), WPBS (WPBS), WSKG Public TV (WSKG), WXXI (WXXI), WPSU (WPSU), WVIA Public Media Studios (WVIA), WTVI (WTVI), Western Reserve PBS (WNEO), WVIZ/PBS ideastream (WVIZ), KCTS 9 (KCTS), Basin PBS (KPBT), KUHT / Channel 8 (KUHT), KLRN (KLRN), KLRU (KLRU), WTJX Channel 12 (WTJX), WCVE PBS (WCVE), KBTC Public Television (KBTC)
|
||||||
- **PearVideo**
|
- **PearVideo**
|
||||||
@@ -748,6 +750,7 @@
|
|||||||
- **play.fm**
|
- **play.fm**
|
||||||
- **player.sky.it**
|
- **player.sky.it**
|
||||||
- **PlayPlusTV**
|
- **PlayPlusTV**
|
||||||
|
- **PlayStuff**
|
||||||
- **PlaysTV**
|
- **PlaysTV**
|
||||||
- **Playtvak**: Playtvak.cz, iDNES.cz and Lidovky.cz
|
- **Playtvak**: Playtvak.cz, iDNES.cz and Lidovky.cz
|
||||||
- **Playvid**
|
- **Playvid**
|
||||||
|
|||||||
+33
-34
@@ -1,41 +1,40 @@
|
|||||||
{
|
{
|
||||||
"consoletitle": false,
|
"consoletitle": false,
|
||||||
"continuedl": true,
|
"continuedl": true,
|
||||||
"forcedescription": false,
|
"forcedescription": false,
|
||||||
"forcefilename": false,
|
"forcefilename": false,
|
||||||
"forceformat": false,
|
"forceformat": false,
|
||||||
"forcethumbnail": false,
|
"forcethumbnail": false,
|
||||||
"forcetitle": false,
|
"forcetitle": false,
|
||||||
"forceurl": false,
|
"forceurl": false,
|
||||||
"force_write_download_archive": false,
|
"force_write_download_archive": false,
|
||||||
"format": "best",
|
"format": "best",
|
||||||
"ignoreerrors": false,
|
"ignoreerrors": false,
|
||||||
"listformats": null,
|
"listformats": null,
|
||||||
"logtostderr": false,
|
"logtostderr": false,
|
||||||
"matchtitle": null,
|
"matchtitle": null,
|
||||||
"max_downloads": null,
|
"max_downloads": null,
|
||||||
"overwrites": null,
|
"overwrites": null,
|
||||||
"nopart": false,
|
"nopart": false,
|
||||||
"noprogress": false,
|
"noprogress": false,
|
||||||
"outtmpl": "%(id)s.%(ext)s",
|
"outtmpl": "%(id)s.%(ext)s",
|
||||||
"password": null,
|
"password": null,
|
||||||
"playlistend": -1,
|
"playliststart": 1,
|
||||||
"playliststart": 1,
|
"prefer_free_formats": false,
|
||||||
"prefer_free_formats": false,
|
"quiet": false,
|
||||||
"quiet": false,
|
"ratelimit": null,
|
||||||
"ratelimit": null,
|
"rejecttitle": null,
|
||||||
"rejecttitle": null,
|
"retries": 10,
|
||||||
"retries": 10,
|
"simulate": false,
|
||||||
"simulate": false,
|
"subtitleslang": null,
|
||||||
"subtitleslang": null,
|
|
||||||
"subtitlesformat": "best",
|
"subtitlesformat": "best",
|
||||||
"test": true,
|
"test": true,
|
||||||
"updatetime": true,
|
"updatetime": true,
|
||||||
"usenetrc": false,
|
"usenetrc": false,
|
||||||
"username": null,
|
"username": null,
|
||||||
"verbose": true,
|
"verbose": true,
|
||||||
"writedescription": false,
|
"writedescription": false,
|
||||||
"writeinfojson": true,
|
"writeinfojson": true,
|
||||||
"writeannotations": false,
|
"writeannotations": false,
|
||||||
"writelink": false,
|
"writelink": false,
|
||||||
"writeurllink": false,
|
"writeurllink": false,
|
||||||
|
|||||||
@@ -29,6 +29,7 @@ class YDL(FakeYDL):
|
|||||||
self.msgs = []
|
self.msgs = []
|
||||||
|
|
||||||
def process_info(self, info_dict):
|
def process_info(self, info_dict):
|
||||||
|
info_dict.pop('__original_infodict', None)
|
||||||
self.downloaded_info_dicts.append(info_dict)
|
self.downloaded_info_dicts.append(info_dict)
|
||||||
|
|
||||||
def to_screen(self, msg):
|
def to_screen(self, msg):
|
||||||
|
|||||||
@@ -121,6 +121,7 @@ def generator(test_case, tname):
|
|||||||
params['outtmpl'] = tname + '_' + params['outtmpl']
|
params['outtmpl'] = tname + '_' + params['outtmpl']
|
||||||
if is_playlist and 'playlist' not in test_case:
|
if is_playlist and 'playlist' not in test_case:
|
||||||
params.setdefault('extract_flat', 'in_playlist')
|
params.setdefault('extract_flat', 'in_playlist')
|
||||||
|
params.setdefault('playlistend', test_case.get('playlist_mincount'))
|
||||||
params.setdefault('skip_download', True)
|
params.setdefault('skip_download', True)
|
||||||
|
|
||||||
ydl = YoutubeDL(params, auto_init=False)
|
ydl = YoutubeDL(params, auto_init=False)
|
||||||
|
|||||||
+118
-67
@@ -48,7 +48,6 @@ from .utils import (
|
|||||||
date_from_str,
|
date_from_str,
|
||||||
DateRange,
|
DateRange,
|
||||||
DEFAULT_OUTTMPL,
|
DEFAULT_OUTTMPL,
|
||||||
OUTTMPL_TYPES,
|
|
||||||
determine_ext,
|
determine_ext,
|
||||||
determine_protocol,
|
determine_protocol,
|
||||||
DOT_DESKTOP_LINK_TEMPLATE,
|
DOT_DESKTOP_LINK_TEMPLATE,
|
||||||
@@ -57,8 +56,8 @@ from .utils import (
|
|||||||
DownloadError,
|
DownloadError,
|
||||||
encode_compat_str,
|
encode_compat_str,
|
||||||
encodeFilename,
|
encodeFilename,
|
||||||
error_to_compat_str,
|
|
||||||
EntryNotInPlaylist,
|
EntryNotInPlaylist,
|
||||||
|
error_to_compat_str,
|
||||||
ExistingVideoReached,
|
ExistingVideoReached,
|
||||||
expand_path,
|
expand_path,
|
||||||
ExtractorError,
|
ExtractorError,
|
||||||
@@ -77,6 +76,7 @@ from .utils import (
|
|||||||
MaxDownloadsReached,
|
MaxDownloadsReached,
|
||||||
network_exceptions,
|
network_exceptions,
|
||||||
orderedSet,
|
orderedSet,
|
||||||
|
OUTTMPL_TYPES,
|
||||||
PagedList,
|
PagedList,
|
||||||
parse_filesize,
|
parse_filesize,
|
||||||
PerRequestProxyHandler,
|
PerRequestProxyHandler,
|
||||||
@@ -84,11 +84,12 @@ from .utils import (
|
|||||||
PostProcessingError,
|
PostProcessingError,
|
||||||
preferredencoding,
|
preferredencoding,
|
||||||
prepend_extension,
|
prepend_extension,
|
||||||
|
process_communicate_or_kill,
|
||||||
random_uuidv4,
|
random_uuidv4,
|
||||||
register_socks_protocols,
|
register_socks_protocols,
|
||||||
|
RejectedVideoReached,
|
||||||
render_table,
|
render_table,
|
||||||
replace_extension,
|
replace_extension,
|
||||||
RejectedVideoReached,
|
|
||||||
SameFileError,
|
SameFileError,
|
||||||
sanitize_filename,
|
sanitize_filename,
|
||||||
sanitize_path,
|
sanitize_path,
|
||||||
@@ -109,7 +110,6 @@ from .utils import (
|
|||||||
YoutubeDLCookieProcessor,
|
YoutubeDLCookieProcessor,
|
||||||
YoutubeDLHandler,
|
YoutubeDLHandler,
|
||||||
YoutubeDLRedirectHandler,
|
YoutubeDLRedirectHandler,
|
||||||
process_communicate_or_kill,
|
|
||||||
)
|
)
|
||||||
from .cache import Cache
|
from .cache import Cache
|
||||||
from .extractor import (
|
from .extractor import (
|
||||||
@@ -177,13 +177,14 @@ class YoutubeDL(object):
|
|||||||
verbose: Print additional info to stdout.
|
verbose: Print additional info to stdout.
|
||||||
quiet: Do not print messages to stdout.
|
quiet: Do not print messages to stdout.
|
||||||
no_warnings: Do not print out anything for warnings.
|
no_warnings: Do not print out anything for warnings.
|
||||||
forceurl: Force printing final URL.
|
forceprint: A list of templates to force print
|
||||||
forcetitle: Force printing title.
|
forceurl: Force printing final URL. (Deprecated)
|
||||||
forceid: Force printing ID.
|
forcetitle: Force printing title. (Deprecated)
|
||||||
forcethumbnail: Force printing thumbnail URL.
|
forceid: Force printing ID. (Deprecated)
|
||||||
forcedescription: Force printing description.
|
forcethumbnail: Force printing thumbnail URL. (Deprecated)
|
||||||
forcefilename: Force printing final filename.
|
forcedescription: Force printing description. (Deprecated)
|
||||||
forceduration: Force printing duration.
|
forcefilename: Force printing final filename. (Deprecated)
|
||||||
|
forceduration: Force printing duration. (Deprecated)
|
||||||
forcejson: Force printing info_dict as JSON.
|
forcejson: Force printing info_dict as JSON.
|
||||||
dump_single_json: Force printing the info_dict of the whole playlist
|
dump_single_json: Force printing the info_dict of the whole playlist
|
||||||
(or video) as a single JSON line.
|
(or video) as a single JSON line.
|
||||||
@@ -386,7 +387,7 @@ class YoutubeDL(object):
|
|||||||
if True, otherwise use ffmpeg/avconv if False, otherwise
|
if True, otherwise use ffmpeg/avconv if False, otherwise
|
||||||
use downloader suggested by extractor if None.
|
use downloader suggested by extractor if None.
|
||||||
compat_opts: Compatibility options. See "Differences in default behavior".
|
compat_opts: Compatibility options. See "Differences in default behavior".
|
||||||
Note that only format-sort, format-spec, no-live-chat,
|
Note that only format-sort, format-spec, no-live-chat, no-attach-info-json
|
||||||
playlist-index, list-formats, no-youtube-channel-redirect
|
playlist-index, list-formats, no-youtube-channel-redirect
|
||||||
and no-youtube-unavailable-videos works when used via the API
|
and no-youtube-unavailable-videos works when used via the API
|
||||||
|
|
||||||
@@ -542,8 +543,7 @@ class YoutubeDL(object):
|
|||||||
def preload_download_archive(fn):
|
def preload_download_archive(fn):
|
||||||
if fn is None:
|
if fn is None:
|
||||||
return False
|
return False
|
||||||
if self.params.get('verbose'):
|
self.write_debug('Loading archive file %r\n' % fn)
|
||||||
self._write_string('[debug] Loading archive file %r\n' % fn)
|
|
||||||
try:
|
try:
|
||||||
with locked_file(fn, 'r', encoding='utf-8') as archive_file:
|
with locked_file(fn, 'r', encoding='utf-8') as archive_file:
|
||||||
for line in archive_file:
|
for line in archive_file:
|
||||||
@@ -649,17 +649,11 @@ class YoutubeDL(object):
|
|||||||
for _ in range(line_count))
|
for _ in range(line_count))
|
||||||
return res[:-len('\n')]
|
return res[:-len('\n')]
|
||||||
|
|
||||||
def to_screen(self, message, skip_eol=False):
|
|
||||||
"""Print message to stdout if not in quiet mode."""
|
|
||||||
return self.to_stdout(
|
|
||||||
message, skip_eol,
|
|
||||||
quiet=self.params.get('quiet', False))
|
|
||||||
|
|
||||||
def _write_string(self, s, out=None):
|
def _write_string(self, s, out=None):
|
||||||
write_string(s, out=out, encoding=self.params.get('encoding'))
|
write_string(s, out=out, encoding=self.params.get('encoding'))
|
||||||
|
|
||||||
def to_stdout(self, message, skip_eol=False, quiet=False):
|
def to_stdout(self, message, skip_eol=False, quiet=False):
|
||||||
"""Print message to stdout if not in quiet mode."""
|
"""Print message to stdout"""
|
||||||
if self.params.get('logger'):
|
if self.params.get('logger'):
|
||||||
self.params['logger'].debug(message)
|
self.params['logger'].debug(message)
|
||||||
elif not quiet:
|
elif not quiet:
|
||||||
@@ -670,7 +664,7 @@ class YoutubeDL(object):
|
|||||||
self._write_string(output, self._screen_file)
|
self._write_string(output, self._screen_file)
|
||||||
|
|
||||||
def to_stderr(self, message):
|
def to_stderr(self, message):
|
||||||
"""Print message to stderr."""
|
"""Print message to stderr"""
|
||||||
assert isinstance(message, compat_str)
|
assert isinstance(message, compat_str)
|
||||||
if self.params.get('logger'):
|
if self.params.get('logger'):
|
||||||
self.params['logger'].error(message)
|
self.params['logger'].error(message)
|
||||||
@@ -748,6 +742,11 @@ class YoutubeDL(object):
|
|||||||
raise DownloadError(message, exc_info)
|
raise DownloadError(message, exc_info)
|
||||||
self._download_retcode = 1
|
self._download_retcode = 1
|
||||||
|
|
||||||
|
def to_screen(self, message, skip_eol=False):
|
||||||
|
"""Print message to stdout if not in quiet mode"""
|
||||||
|
self.to_stdout(
|
||||||
|
message, skip_eol, quiet=self.params.get('quiet', False))
|
||||||
|
|
||||||
def report_warning(self, message):
|
def report_warning(self, message):
|
||||||
'''
|
'''
|
||||||
Print the message to stderr, it will be prefixed with 'WARNING:'
|
Print the message to stderr, it will be prefixed with 'WARNING:'
|
||||||
@@ -777,6 +776,16 @@ class YoutubeDL(object):
|
|||||||
error_message = '%s %s' % (_msg_header, message)
|
error_message = '%s %s' % (_msg_header, message)
|
||||||
self.trouble(error_message, tb)
|
self.trouble(error_message, tb)
|
||||||
|
|
||||||
|
def write_debug(self, message):
|
||||||
|
'''Log debug message or Print message to stderr'''
|
||||||
|
if not self.params.get('verbose', False):
|
||||||
|
return
|
||||||
|
message = '[debug] %s' % message
|
||||||
|
if self.params.get('logger'):
|
||||||
|
self.params['logger'].debug(message)
|
||||||
|
else:
|
||||||
|
self._write_string('%s\n' % message)
|
||||||
|
|
||||||
def report_file_already_downloaded(self, file_name):
|
def report_file_already_downloaded(self, file_name):
|
||||||
"""Report file has already been fully downloaded."""
|
"""Report file has already been fully downloaded."""
|
||||||
try:
|
try:
|
||||||
@@ -812,7 +821,7 @@ class YoutubeDL(object):
|
|||||||
|
|
||||||
# duration_string
|
# duration_string
|
||||||
template_dict['duration_string'] = ( # %(duration>%H-%M-%S)s is wrong if duration > 24hrs
|
template_dict['duration_string'] = ( # %(duration>%H-%M-%S)s is wrong if duration > 24hrs
|
||||||
formatSeconds(info_dict['duration'], '-')
|
formatSeconds(info_dict['duration'], '-' if sanitize else ':')
|
||||||
if info_dict.get('duration', None) is not None
|
if info_dict.get('duration', None) is not None
|
||||||
else None)
|
else None)
|
||||||
|
|
||||||
@@ -1178,14 +1187,29 @@ class YoutubeDL(object):
|
|||||||
|
|
||||||
if result_type == 'video':
|
if result_type == 'video':
|
||||||
self.add_extra_info(ie_result, extra_info)
|
self.add_extra_info(ie_result, extra_info)
|
||||||
return self.process_video_result(ie_result, download=download)
|
ie_result = self.process_video_result(ie_result, download=download)
|
||||||
|
additional_urls = (ie_result or {}).get('additional_urls')
|
||||||
|
if additional_urls:
|
||||||
|
# TODO: Improve MetadataFromFieldPP to allow setting a list
|
||||||
|
if isinstance(additional_urls, compat_str):
|
||||||
|
additional_urls = [additional_urls]
|
||||||
|
self.to_screen(
|
||||||
|
'[info] %s: %d additional URL(s) requested' % (ie_result['id'], len(additional_urls)))
|
||||||
|
self.write_debug('Additional URLs: "%s"' % '", "'.join(additional_urls))
|
||||||
|
ie_result['additional_entries'] = [
|
||||||
|
self.extract_info(
|
||||||
|
url, download, extra_info,
|
||||||
|
force_generic_extractor=self.params.get('force_generic_extractor'))
|
||||||
|
for url in additional_urls
|
||||||
|
]
|
||||||
|
return ie_result
|
||||||
elif result_type == 'url':
|
elif result_type == 'url':
|
||||||
# We have to add extra_info to the results because it may be
|
# We have to add extra_info to the results because it may be
|
||||||
# contained in a playlist
|
# contained in a playlist
|
||||||
return self.extract_info(ie_result['url'],
|
return self.extract_info(
|
||||||
download,
|
ie_result['url'], download,
|
||||||
ie_key=ie_result.get('ie_key'),
|
ie_key=ie_result.get('ie_key'),
|
||||||
extra_info=extra_info)
|
extra_info=extra_info)
|
||||||
elif result_type == 'url_transparent':
|
elif result_type == 'url_transparent':
|
||||||
# Use the information from the embedding page
|
# Use the information from the embedding page
|
||||||
info = self.extract_info(
|
info = self.extract_info(
|
||||||
@@ -1370,6 +1394,9 @@ class YoutubeDL(object):
|
|||||||
except (OSError, IOError):
|
except (OSError, IOError):
|
||||||
self.report_error('Cannot write playlist metadata to JSON file ' + infofn)
|
self.report_error('Cannot write playlist metadata to JSON file ' + infofn)
|
||||||
|
|
||||||
|
# TODO: This should be passed to ThumbnailsConvertor if necessary
|
||||||
|
self._write_thumbnails(ie_copy, self.prepare_filename(ie_copy, 'pl_thumbnail'))
|
||||||
|
|
||||||
if self.params.get('writedescription', False):
|
if self.params.get('writedescription', False):
|
||||||
descfn = self.prepare_filename(ie_copy, 'pl_description')
|
descfn = self.prepare_filename(ie_copy, 'pl_description')
|
||||||
if not self._ensure_dir_exists(encodeFilename(descfn)):
|
if not self._ensure_dir_exists(encodeFilename(descfn)):
|
||||||
@@ -1717,9 +1744,13 @@ class YoutubeDL(object):
|
|||||||
expand_path(paths.get('home', '').strip()),
|
expand_path(paths.get('home', '').strip()),
|
||||||
expand_path(paths.get('temp', '').strip()),
|
expand_path(paths.get('temp', '').strip()),
|
||||||
'ytdl.%s.f%s.check-format' % (random_uuidv4(), f['format_id']))
|
'ytdl.%s.f%s.check-format' % (random_uuidv4(), f['format_id']))
|
||||||
dl, _ = self.dl(temp_file, f, test=True)
|
try:
|
||||||
if os.path.exists(temp_file):
|
dl, _ = self.dl(temp_file, f, test=True)
|
||||||
os.remove(temp_file)
|
except (ExtractorError, IOError, OSError, ValueError) + network_exceptions:
|
||||||
|
dl = False
|
||||||
|
finally:
|
||||||
|
if os.path.exists(temp_file):
|
||||||
|
os.remove(temp_file)
|
||||||
if dl:
|
if dl:
|
||||||
yield f
|
yield f
|
||||||
else:
|
else:
|
||||||
@@ -1928,7 +1959,8 @@ class YoutubeDL(object):
|
|||||||
t.get('preference') if t.get('preference') is not None else -1,
|
t.get('preference') if t.get('preference') is not None else -1,
|
||||||
t.get('width') if t.get('width') is not None else -1,
|
t.get('width') if t.get('width') is not None else -1,
|
||||||
t.get('height') if t.get('height') is not None else -1,
|
t.get('height') if t.get('height') is not None else -1,
|
||||||
t.get('id') if t.get('id') is not None else '', t.get('url')))
|
t.get('id') if t.get('id') is not None else '',
|
||||||
|
t.get('url')))
|
||||||
for i, t in enumerate(thumbnails):
|
for i, t in enumerate(thumbnails):
|
||||||
t['url'] = sanitize_url(t['url'])
|
t['url'] = sanitize_url(t['url'])
|
||||||
if t.get('width') and t.get('height'):
|
if t.get('width') and t.get('height'):
|
||||||
@@ -2072,6 +2104,9 @@ class YoutubeDL(object):
|
|||||||
# element in the 'formats' field in info_dict is info_dict itself,
|
# element in the 'formats' field in info_dict is info_dict itself,
|
||||||
# which can't be exported to json
|
# which can't be exported to json
|
||||||
info_dict['formats'] = formats
|
info_dict['formats'] = formats
|
||||||
|
|
||||||
|
info_dict, _ = self.pre_process(info_dict)
|
||||||
|
|
||||||
if self.params.get('listformats'):
|
if self.params.get('listformats'):
|
||||||
if not info_dict.get('formats'):
|
if not info_dict.get('formats'):
|
||||||
raise ExtractorError('No video formats found', expected=True)
|
raise ExtractorError('No video formats found', expected=True)
|
||||||
@@ -2081,8 +2116,7 @@ class YoutubeDL(object):
|
|||||||
req_format = self.params.get('format')
|
req_format = self.params.get('format')
|
||||||
if req_format is None:
|
if req_format is None:
|
||||||
req_format = self._default_format_spec(info_dict, download=download)
|
req_format = self._default_format_spec(info_dict, download=download)
|
||||||
if self.params.get('verbose'):
|
self.write_debug('Default format spec: %s' % req_format)
|
||||||
self.to_screen('[debug] Default format spec: %s' % req_format)
|
|
||||||
|
|
||||||
format_selector = self.build_format_selector(req_format)
|
format_selector = self.build_format_selector(req_format)
|
||||||
|
|
||||||
@@ -2120,14 +2154,13 @@ class YoutubeDL(object):
|
|||||||
self.report_warning('Requested format is not available')
|
self.report_warning('Requested format is not available')
|
||||||
elif download:
|
elif download:
|
||||||
self.to_screen(
|
self.to_screen(
|
||||||
'[info] %s: Downloading format(s) %s'
|
'[info] %s: Downloading %d format(s): %s' % (
|
||||||
% (info_dict['id'], ", ".join([f['format_id'] for f in formats_to_download])))
|
info_dict['id'], len(formats_to_download),
|
||||||
if len(formats_to_download) > 1:
|
", ".join([f['format_id'] for f in formats_to_download])))
|
||||||
self.to_screen(
|
|
||||||
'[info] %s: Downloading video in %s formats'
|
|
||||||
% (info_dict['id'], len(formats_to_download)))
|
|
||||||
for fmt in formats_to_download:
|
for fmt in formats_to_download:
|
||||||
new_info = dict(info_dict)
|
new_info = dict(info_dict)
|
||||||
|
# Save a reference to the original info_dict so that it can be modified in process_info if needed
|
||||||
|
new_info['__original_infodict'] = info_dict
|
||||||
new_info.update(fmt)
|
new_info.update(fmt)
|
||||||
self.process_info(new_info)
|
self.process_info(new_info)
|
||||||
# We update the info dict with the best quality format (backwards compatibility)
|
# We update the info dict with the best quality format (backwards compatibility)
|
||||||
@@ -2172,6 +2205,7 @@ class YoutubeDL(object):
|
|||||||
requested_langs = ['en']
|
requested_langs = ['en']
|
||||||
else:
|
else:
|
||||||
requested_langs = [list(all_sub_langs)[0]]
|
requested_langs = [list(all_sub_langs)[0]]
|
||||||
|
self.write_debug('Downloading subtitles: %s' % ', '.join(requested_langs))
|
||||||
|
|
||||||
formats_query = self.params.get('subtitlesformat', 'best')
|
formats_query = self.params.get('subtitlesformat', 'best')
|
||||||
formats_preference = formats_query.split('/') if formats_query else []
|
formats_preference = formats_query.split('/') if formats_query else []
|
||||||
@@ -2198,32 +2232,43 @@ class YoutubeDL(object):
|
|||||||
return subs
|
return subs
|
||||||
|
|
||||||
def __forced_printings(self, info_dict, filename, incomplete):
|
def __forced_printings(self, info_dict, filename, incomplete):
|
||||||
def print_mandatory(field):
|
def print_mandatory(field, actual_field=None):
|
||||||
|
if actual_field is None:
|
||||||
|
actual_field = field
|
||||||
if (self.params.get('force%s' % field, False)
|
if (self.params.get('force%s' % field, False)
|
||||||
and (not incomplete or info_dict.get(field) is not None)):
|
and (not incomplete or info_dict.get(actual_field) is not None)):
|
||||||
self.to_stdout(info_dict[field])
|
self.to_stdout(info_dict[actual_field])
|
||||||
|
|
||||||
def print_optional(field):
|
def print_optional(field):
|
||||||
if (self.params.get('force%s' % field, False)
|
if (self.params.get('force%s' % field, False)
|
||||||
and info_dict.get(field) is not None):
|
and info_dict.get(field) is not None):
|
||||||
self.to_stdout(info_dict[field])
|
self.to_stdout(info_dict[field])
|
||||||
|
|
||||||
|
info_dict = info_dict.copy()
|
||||||
|
if filename is not None:
|
||||||
|
info_dict['filename'] = filename
|
||||||
|
if info_dict.get('requested_formats') is not None:
|
||||||
|
# For RTMP URLs, also include the playpath
|
||||||
|
info_dict['urls'] = '\n'.join(f['url'] + f.get('play_path', '') for f in info_dict['requested_formats'])
|
||||||
|
elif 'url' in info_dict:
|
||||||
|
info_dict['urls'] = info_dict['url'] + info_dict.get('play_path', '')
|
||||||
|
|
||||||
|
for tmpl in self.params.get('forceprint', []):
|
||||||
|
if re.match(r'\w+$', tmpl):
|
||||||
|
tmpl = '%({})s'.format(tmpl)
|
||||||
|
tmpl, info_copy = self.prepare_outtmpl(tmpl, info_dict)
|
||||||
|
self.to_stdout(tmpl % info_copy)
|
||||||
|
|
||||||
print_mandatory('title')
|
print_mandatory('title')
|
||||||
print_mandatory('id')
|
print_mandatory('id')
|
||||||
if self.params.get('forceurl', False) and not incomplete:
|
print_mandatory('url', 'urls')
|
||||||
if info_dict.get('requested_formats') is not None:
|
|
||||||
for f in info_dict['requested_formats']:
|
|
||||||
self.to_stdout(f['url'] + f.get('play_path', ''))
|
|
||||||
else:
|
|
||||||
# For RTMP URLs, also include the playpath
|
|
||||||
self.to_stdout(info_dict['url'] + info_dict.get('play_path', ''))
|
|
||||||
print_optional('thumbnail')
|
print_optional('thumbnail')
|
||||||
print_optional('description')
|
print_optional('description')
|
||||||
if self.params.get('forcefilename', False) and filename is not None:
|
print_optional('filename')
|
||||||
self.to_stdout(filename)
|
|
||||||
if self.params.get('forceduration', False) and info_dict.get('duration') is not None:
|
if self.params.get('forceduration', False) and info_dict.get('duration') is not None:
|
||||||
self.to_stdout(formatSeconds(info_dict['duration']))
|
self.to_stdout(formatSeconds(info_dict['duration']))
|
||||||
print_mandatory('format')
|
print_mandatory('format')
|
||||||
|
|
||||||
if self.params.get('forcejson', False):
|
if self.params.get('forcejson', False):
|
||||||
self.post_extract(info_dict)
|
self.post_extract(info_dict)
|
||||||
self.to_stdout(json.dumps(info_dict, default=repr))
|
self.to_stdout(json.dumps(info_dict, default=repr))
|
||||||
@@ -2249,8 +2294,7 @@ class YoutubeDL(object):
|
|||||||
if not test:
|
if not test:
|
||||||
for ph in self._progress_hooks:
|
for ph in self._progress_hooks:
|
||||||
fd.add_progress_hook(ph)
|
fd.add_progress_hook(ph)
|
||||||
if self.params.get('verbose'):
|
self.write_debug('Invoking downloader on %r' % info.get('url'))
|
||||||
self.to_screen('[debug] Invoking downloader on %r' % info.get('url'))
|
|
||||||
new_info = dict(info)
|
new_info = dict(info)
|
||||||
if new_info.get('http_headers') is None:
|
if new_info.get('http_headers') is None:
|
||||||
new_info['http_headers'] = self._calc_headers(new_info)
|
new_info['http_headers'] = self._calc_headers(new_info)
|
||||||
@@ -2280,8 +2324,6 @@ class YoutubeDL(object):
|
|||||||
self.post_extract(info_dict)
|
self.post_extract(info_dict)
|
||||||
self._num_downloads += 1
|
self._num_downloads += 1
|
||||||
|
|
||||||
info_dict, _ = self.pre_process(info_dict)
|
|
||||||
|
|
||||||
# info_dict['_filename'] needs to be set for backward compatibility
|
# info_dict['_filename'] needs to be set for backward compatibility
|
||||||
info_dict['_filename'] = full_filename = self.prepare_filename(info_dict, warn=True)
|
info_dict['_filename'] = full_filename = self.prepare_filename(info_dict, warn=True)
|
||||||
temp_filename = self.prepare_filename(info_dict, 'temp')
|
temp_filename = self.prepare_filename(info_dict, 'temp')
|
||||||
@@ -2376,7 +2418,7 @@ class YoutubeDL(object):
|
|||||||
self.dl(sub_filename, sub_info.copy(), subtitle=True)
|
self.dl(sub_filename, sub_info.copy(), subtitle=True)
|
||||||
sub_info['filepath'] = sub_filename
|
sub_info['filepath'] = sub_filename
|
||||||
files_to_move[sub_filename] = sub_filename_final
|
files_to_move[sub_filename] = sub_filename_final
|
||||||
except tuple([ExtractorError, IOError, OSError, ValueError] + network_exceptions) as err:
|
except (ExtractorError, IOError, OSError, ValueError) + network_exceptions as err:
|
||||||
self.report_warning('Unable to download subtitle for "%s": %s' %
|
self.report_warning('Unable to download subtitle for "%s": %s' %
|
||||||
(sub_lang, error_to_compat_str(err)))
|
(sub_lang, error_to_compat_str(err)))
|
||||||
continue
|
continue
|
||||||
@@ -2719,6 +2761,7 @@ class YoutubeDL(object):
|
|||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def filter_requested_info(info_dict, actually_filter=True):
|
def filter_requested_info(info_dict, actually_filter=True):
|
||||||
|
info_dict.pop('__original_infodict', None) # Always remove this
|
||||||
if not actually_filter:
|
if not actually_filter:
|
||||||
info_dict['epoch'] = int(time.time())
|
info_dict['epoch'] = int(time.time())
|
||||||
return info_dict
|
return info_dict
|
||||||
@@ -2763,13 +2806,14 @@ class YoutubeDL(object):
|
|||||||
actual_post_extract(video_dict or {})
|
actual_post_extract(video_dict or {})
|
||||||
return
|
return
|
||||||
|
|
||||||
if '__post_extractor' not in info_dict:
|
post_extractor = info_dict.get('__post_extractor') or (lambda: {})
|
||||||
return
|
extra = post_extractor().items()
|
||||||
post_extractor = info_dict['__post_extractor']
|
info_dict.update(extra)
|
||||||
if post_extractor:
|
info_dict.pop('__post_extractor', None)
|
||||||
info_dict.update(post_extractor().items())
|
|
||||||
del info_dict['__post_extractor']
|
original_infodict = info_dict.get('__original_infodict') or {}
|
||||||
return
|
original_infodict.update(extra)
|
||||||
|
original_infodict.pop('__post_extractor', None)
|
||||||
|
|
||||||
actual_post_extract(info_dict or {})
|
actual_post_extract(info_dict or {})
|
||||||
|
|
||||||
@@ -2982,10 +3026,17 @@ class YoutubeDL(object):
|
|||||||
return
|
return
|
||||||
self.to_screen(
|
self.to_screen(
|
||||||
'Available %s for %s:' % (name, video_id))
|
'Available %s for %s:' % (name, video_id))
|
||||||
|
|
||||||
|
def _row(lang, formats):
|
||||||
|
exts, names = zip(*((f['ext'], f.get('name', 'unknown')) for f in reversed(formats)))
|
||||||
|
if len(set(names)) == 1:
|
||||||
|
names = [] if names[0] == 'unknown' else names[:1]
|
||||||
|
return [lang, ', '.join(names), ', '.join(exts)]
|
||||||
|
|
||||||
self.to_screen(render_table(
|
self.to_screen(render_table(
|
||||||
['Language', 'formats'],
|
['Language', 'Name', 'Formats'],
|
||||||
[[lang, ', '.join(f['ext'] for f in reversed(formats))]
|
[_row(lang, formats) for lang, formats in subtitles.items()],
|
||||||
for lang, formats in subtitles.items()]))
|
hideEmpty=True))
|
||||||
|
|
||||||
def urlopen(self, req):
|
def urlopen(self, req):
|
||||||
""" Start an HTTP download """
|
""" Start an HTTP download """
|
||||||
|
|||||||
+3
-2
@@ -260,7 +260,7 @@ def _real_main(argv=None):
|
|||||||
all_compat_opts = [
|
all_compat_opts = [
|
||||||
'filename', 'format-sort', 'abort-on-error', 'format-spec', 'multistreams',
|
'filename', 'format-sort', 'abort-on-error', 'format-spec', 'multistreams',
|
||||||
'no-playlist-metafiles', 'no-live-chat', 'playlist-index', 'list-formats',
|
'no-playlist-metafiles', 'no-live-chat', 'playlist-index', 'list-formats',
|
||||||
'no-youtube-channel-redirect', 'no-youtube-unavailable-videos',
|
'no-youtube-channel-redirect', 'no-youtube-unavailable-videos', 'no-attach-info-json',
|
||||||
]
|
]
|
||||||
compat_opts = parse_compat_opts()
|
compat_opts = parse_compat_opts()
|
||||||
|
|
||||||
@@ -321,7 +321,7 @@ def _real_main(argv=None):
|
|||||||
if re.match(MetadataFromFieldPP.regex, f) is None:
|
if re.match(MetadataFromFieldPP.regex, f) is None:
|
||||||
parser.error('invalid format string "%s" specified for --parse-metadata' % f)
|
parser.error('invalid format string "%s" specified for --parse-metadata' % f)
|
||||||
|
|
||||||
any_getting = opts.geturl or opts.gettitle or opts.getid or opts.getthumbnail or opts.getdescription or opts.getfilename or opts.getformat or opts.getduration or opts.dumpjson or opts.dump_single_json
|
any_getting = opts.forceprint or opts.geturl or opts.gettitle or opts.getid or opts.getthumbnail or opts.getdescription or opts.getfilename or opts.getformat or opts.getduration or opts.dumpjson or opts.dump_single_json
|
||||||
any_printing = opts.print_json
|
any_printing = opts.print_json
|
||||||
download_archive_fn = expand_path(opts.download_archive) if opts.download_archive is not None else opts.download_archive
|
download_archive_fn = expand_path(opts.download_archive) if opts.download_archive is not None else opts.download_archive
|
||||||
|
|
||||||
@@ -508,6 +508,7 @@ def _real_main(argv=None):
|
|||||||
'forceduration': opts.getduration,
|
'forceduration': opts.getduration,
|
||||||
'forcefilename': opts.getfilename,
|
'forcefilename': opts.getfilename,
|
||||||
'forceformat': opts.getformat,
|
'forceformat': opts.getformat,
|
||||||
|
'forceprint': opts.forceprint,
|
||||||
'forcejson': opts.dumpjson or opts.print_json,
|
'forcejson': opts.dumpjson or opts.print_json,
|
||||||
'dump_single_json': opts.dump_single_json,
|
'dump_single_json': opts.dump_single_json,
|
||||||
'force_write_download_archive': opts.force_write_download_archive,
|
'force_write_download_archive': opts.force_write_download_archive,
|
||||||
|
|||||||
@@ -164,6 +164,9 @@ class FileDownloader(object):
|
|||||||
def report_error(self, *args, **kargs):
|
def report_error(self, *args, **kargs):
|
||||||
self.ydl.report_error(*args, **kargs)
|
self.ydl.report_error(*args, **kargs)
|
||||||
|
|
||||||
|
def write_debug(self, *args, **kargs):
|
||||||
|
self.ydl.write_debug(*args, **kargs)
|
||||||
|
|
||||||
def slow_down(self, start_time, now, byte_counter):
|
def slow_down(self, start_time, now, byte_counter):
|
||||||
"""Sleep if the download speed is over the rate limit."""
|
"""Sleep if the download speed is over the rate limit."""
|
||||||
rate_limit = self.params.get('ratelimit')
|
rate_limit = self.params.get('ratelimit')
|
||||||
@@ -402,5 +405,4 @@ class FileDownloader(object):
|
|||||||
if exe is None:
|
if exe is None:
|
||||||
exe = os.path.basename(str_args[0])
|
exe = os.path.basename(str_args[0])
|
||||||
|
|
||||||
self.to_screen('[debug] %s command line: %s' % (
|
self.write_debug('%s command line: %s' % (exe, shell_quote(str_args)))
|
||||||
exe, shell_quote(str_args)))
|
|
||||||
|
|||||||
@@ -1414,7 +1414,7 @@ class AdobePassIE(InfoExtractor):
|
|||||||
authn_token = None
|
authn_token = None
|
||||||
if not authn_token:
|
if not authn_token:
|
||||||
# TODO add support for other TV Providers
|
# TODO add support for other TV Providers
|
||||||
mso_id = self._downloader.params.get('ap_mso')
|
mso_id = self.get_param('ap_mso')
|
||||||
if not mso_id:
|
if not mso_id:
|
||||||
raise_mvpd_required()
|
raise_mvpd_required()
|
||||||
username, password = self._get_login_info('ap_username', 'ap_password', mso_id)
|
username, password = self._get_login_info('ap_username', 'ap_password', mso_id)
|
||||||
|
|||||||
@@ -323,7 +323,7 @@ class AfreecaTVIE(InfoExtractor):
|
|||||||
'url': file_url,
|
'url': file_url,
|
||||||
'format_id': 'http',
|
'format_id': 'http',
|
||||||
}]
|
}]
|
||||||
if not formats and not self._downloader.params.get('ignore_no_formats'):
|
if not formats and not self.get_param('ignore_no_formats'):
|
||||||
continue
|
continue
|
||||||
self._sort_formats(formats)
|
self._sort_formats(formats)
|
||||||
file_info = common_entry.copy()
|
file_info = common_entry.copy()
|
||||||
|
|||||||
@@ -245,3 +245,31 @@ class AudiusPlaylistIE(AudiusBaseIE):
|
|||||||
return self.playlist_result(entries, playlist_id,
|
return self.playlist_result(entries, playlist_id,
|
||||||
playlist_data.get('playlist_name', title),
|
playlist_data.get('playlist_name', title),
|
||||||
playlist_data.get('description'))
|
playlist_data.get('description'))
|
||||||
|
|
||||||
|
|
||||||
|
class AudiusProfileIE(AudiusPlaylistIE):
|
||||||
|
IE_NAME = 'audius:artist'
|
||||||
|
IE_DESC = 'Audius.co profile/artist pages'
|
||||||
|
_VALID_URL = r'https?://(?:www)?audius\.co/(?P<id>[^\/]+)/?(?:[?#]|$)'
|
||||||
|
_TEST = {
|
||||||
|
'url': 'https://audius.co/pzl/',
|
||||||
|
'info_dict': {
|
||||||
|
'id': 'ezRo7',
|
||||||
|
'description': 'TAMALE\n\nContact: officialpzl@gmail.com',
|
||||||
|
'title': 'pzl',
|
||||||
|
},
|
||||||
|
'playlist_count': 24,
|
||||||
|
}
|
||||||
|
|
||||||
|
def _real_extract(self, url):
|
||||||
|
self._select_api_base()
|
||||||
|
profile_id = self._match_id(url)
|
||||||
|
try:
|
||||||
|
_profile_data = self._api_request('/full/users/handle/' + profile_id, profile_id)
|
||||||
|
except ExtractorError as e:
|
||||||
|
raise ExtractorError('Could not download profile info; ' + str(e))
|
||||||
|
profile_audius_id = _profile_data[0]['id']
|
||||||
|
profile_bio = _profile_data[0].get('bio')
|
||||||
|
|
||||||
|
api_call = self._api_request('/full/users/handle/%s/tracks' % profile_id, profile_id)
|
||||||
|
return self.playlist_result(self._build_playlist(api_call), profile_audius_id, profile_id, profile_bio)
|
||||||
|
|||||||
@@ -1271,7 +1271,7 @@ class BBCIE(BBCCoUkIE):
|
|||||||
entries = []
|
entries = []
|
||||||
for num, media_meta in enumerate(medias, start=1):
|
for num, media_meta in enumerate(medias, start=1):
|
||||||
formats, subtitles = self._extract_from_media_meta(media_meta, playlist_id)
|
formats, subtitles = self._extract_from_media_meta(media_meta, playlist_id)
|
||||||
if not formats and not self._downloader.params.get('ignore_no_formats'):
|
if not formats and not self.get_param('ignore_no_formats'):
|
||||||
continue
|
continue
|
||||||
self._sort_formats(formats)
|
self._sort_formats(formats)
|
||||||
|
|
||||||
|
|||||||
@@ -153,7 +153,7 @@ class BiliBiliIE(InfoExtractor):
|
|||||||
# Bilibili anthologies are similar to playlists but all videos share the same video ID as the anthology itself.
|
# Bilibili anthologies are similar to playlists but all videos share the same video ID as the anthology itself.
|
||||||
# If the video has no page argument, check to see if it's an anthology
|
# If the video has no page argument, check to see if it's an anthology
|
||||||
if page_id is None:
|
if page_id is None:
|
||||||
if not self._downloader.params.get('noplaylist'):
|
if not self.get_param('noplaylist'):
|
||||||
r = self._extract_anthology_entries(bv_id, video_id, webpage)
|
r = self._extract_anthology_entries(bv_id, video_id, webpage)
|
||||||
if r is not None:
|
if r is not None:
|
||||||
self.to_screen('Downloading anthology %s - add --no-playlist to just download video' % video_id)
|
self.to_screen('Downloading anthology %s - add --no-playlist to just download video' % video_id)
|
||||||
@@ -299,7 +299,7 @@ class BiliBiliIE(InfoExtractor):
|
|||||||
'tags': tags,
|
'tags': tags,
|
||||||
'raw_tags': raw_tags,
|
'raw_tags': raw_tags,
|
||||||
}
|
}
|
||||||
if self._downloader.params.get('getcomments', False):
|
if self.get_param('getcomments', False):
|
||||||
def get_comments():
|
def get_comments():
|
||||||
comments = self._get_all_comment_pages(video_id)
|
comments = self._get_all_comment_pages(video_id)
|
||||||
return {
|
return {
|
||||||
|
|||||||
@@ -478,7 +478,7 @@ class BrightcoveNewIE(AdobePassIE):
|
|||||||
container = source.get('container')
|
container = source.get('container')
|
||||||
ext = mimetype2ext(source.get('type'))
|
ext = mimetype2ext(source.get('type'))
|
||||||
src = source.get('src')
|
src = source.get('src')
|
||||||
skip_unplayable = not self._downloader.params.get('allow_unplayable_formats')
|
skip_unplayable = not self.get_param('allow_unplayable_formats')
|
||||||
# https://support.brightcove.com/playback-api-video-fields-reference#key_systems_object
|
# https://support.brightcove.com/playback-api-video-fields-reference#key_systems_object
|
||||||
if skip_unplayable and (container == 'WVM' or source.get('key_systems')):
|
if skip_unplayable and (container == 'WVM' or source.get('key_systems')):
|
||||||
num_drm_sources += 1
|
num_drm_sources += 1
|
||||||
@@ -547,7 +547,7 @@ class BrightcoveNewIE(AdobePassIE):
|
|||||||
error = errors[0]
|
error = errors[0]
|
||||||
self.raise_no_formats(
|
self.raise_no_formats(
|
||||||
error.get('message') or error.get('error_subcode') or error['error_code'], expected=True)
|
error.get('message') or error.get('error_subcode') or error['error_code'], expected=True)
|
||||||
elif (not self._downloader.params.get('allow_unplayable_formats')
|
elif (not self.get_param('allow_unplayable_formats')
|
||||||
and sources and num_drm_sources == len(sources)):
|
and sources and num_drm_sources == len(sources)):
|
||||||
raise ExtractorError('This video is DRM protected.', expected=True)
|
raise ExtractorError('This video is DRM protected.', expected=True)
|
||||||
|
|
||||||
|
|||||||
@@ -147,7 +147,7 @@ class CeskaTelevizeIE(InfoExtractor):
|
|||||||
is_live = item.get('type') == 'LIVE'
|
is_live = item.get('type') == 'LIVE'
|
||||||
formats = []
|
formats = []
|
||||||
for format_id, stream_url in item.get('streamUrls', {}).items():
|
for format_id, stream_url in item.get('streamUrls', {}).items():
|
||||||
if (not self._downloader.params.get('allow_unplayable_formats')
|
if (not self.get_param('allow_unplayable_formats')
|
||||||
and 'drmOnly=true' in stream_url):
|
and 'drmOnly=true' in stream_url):
|
||||||
continue
|
continue
|
||||||
if 'playerType=flash' in stream_url:
|
if 'playerType=flash' in stream_url:
|
||||||
|
|||||||
+72
-62
@@ -250,6 +250,8 @@ class InfoExtractor(object):
|
|||||||
entry and one of:
|
entry and one of:
|
||||||
* "data": The subtitles file contents
|
* "data": The subtitles file contents
|
||||||
* "url": A URL pointing to the subtitles file
|
* "url": A URL pointing to the subtitles file
|
||||||
|
It can optionally also have:
|
||||||
|
* "name": Name or description of the subtitles
|
||||||
"ext" will be calculated from URL if missing
|
"ext" will be calculated from URL if missing
|
||||||
automatic_captions: Like 'subtitles'; contains automatically generated
|
automatic_captions: Like 'subtitles'; contains automatically generated
|
||||||
captions instead of normal subtitles
|
captions instead of normal subtitles
|
||||||
@@ -420,6 +422,14 @@ class InfoExtractor(object):
|
|||||||
_GEO_IP_BLOCKS = None
|
_GEO_IP_BLOCKS = None
|
||||||
_WORKING = True
|
_WORKING = True
|
||||||
|
|
||||||
|
_LOGIN_HINTS = {
|
||||||
|
'any': 'Use --cookies, --username and --password or --netrc to provide account credentials',
|
||||||
|
'cookies': (
|
||||||
|
'Use --cookies for the authentication. '
|
||||||
|
'See https://github.com/ytdl-org/youtube-dl#how-do-i-pass-cookies-to-youtube-dl for how to pass cookies'),
|
||||||
|
'password': 'Use --username and --password or --netrc to provide account credentials',
|
||||||
|
}
|
||||||
|
|
||||||
def __init__(self, downloader=None):
|
def __init__(self, downloader=None):
|
||||||
"""Constructor. Receives an optional downloader."""
|
"""Constructor. Receives an optional downloader."""
|
||||||
self._ready = False
|
self._ready = False
|
||||||
@@ -489,7 +499,7 @@ class InfoExtractor(object):
|
|||||||
if not self._x_forwarded_for_ip:
|
if not self._x_forwarded_for_ip:
|
||||||
|
|
||||||
# Geo bypass mechanism is explicitly disabled by user
|
# Geo bypass mechanism is explicitly disabled by user
|
||||||
if not self._downloader.params.get('geo_bypass', True):
|
if not self.get_param('geo_bypass', True):
|
||||||
return
|
return
|
||||||
|
|
||||||
if not geo_bypass_context:
|
if not geo_bypass_context:
|
||||||
@@ -511,7 +521,7 @@ class InfoExtractor(object):
|
|||||||
|
|
||||||
# Explicit IP block specified by user, use it right away
|
# Explicit IP block specified by user, use it right away
|
||||||
# regardless of whether extractor is geo bypassable or not
|
# regardless of whether extractor is geo bypassable or not
|
||||||
ip_block = self._downloader.params.get('geo_bypass_ip_block', None)
|
ip_block = self.get_param('geo_bypass_ip_block', None)
|
||||||
|
|
||||||
# Otherwise use random IP block from geo bypass context but only
|
# Otherwise use random IP block from geo bypass context but only
|
||||||
# if extractor is known as geo bypassable
|
# if extractor is known as geo bypassable
|
||||||
@@ -522,17 +532,15 @@ class InfoExtractor(object):
|
|||||||
|
|
||||||
if ip_block:
|
if ip_block:
|
||||||
self._x_forwarded_for_ip = GeoUtils.random_ipv4(ip_block)
|
self._x_forwarded_for_ip = GeoUtils.random_ipv4(ip_block)
|
||||||
if self._downloader.params.get('verbose', False):
|
self._downloader.write_debug(
|
||||||
self._downloader.to_screen(
|
'[debug] Using fake IP %s as X-Forwarded-For' % self._x_forwarded_for_ip)
|
||||||
'[debug] Using fake IP %s as X-Forwarded-For.'
|
|
||||||
% self._x_forwarded_for_ip)
|
|
||||||
return
|
return
|
||||||
|
|
||||||
# Path 2: bypassing based on country code
|
# Path 2: bypassing based on country code
|
||||||
|
|
||||||
# Explicit country code specified by user, use it right away
|
# Explicit country code specified by user, use it right away
|
||||||
# regardless of whether extractor is geo bypassable or not
|
# regardless of whether extractor is geo bypassable or not
|
||||||
country = self._downloader.params.get('geo_bypass_country', None)
|
country = self.get_param('geo_bypass_country', None)
|
||||||
|
|
||||||
# Otherwise use random country code from geo bypass context but
|
# Otherwise use random country code from geo bypass context but
|
||||||
# only if extractor is known as geo bypassable
|
# only if extractor is known as geo bypassable
|
||||||
@@ -543,10 +551,8 @@ class InfoExtractor(object):
|
|||||||
|
|
||||||
if country:
|
if country:
|
||||||
self._x_forwarded_for_ip = GeoUtils.random_ipv4(country)
|
self._x_forwarded_for_ip = GeoUtils.random_ipv4(country)
|
||||||
if self._downloader.params.get('verbose', False):
|
self._downloader.write_debug(
|
||||||
self._downloader.to_screen(
|
'Using fake IP %s (%s) as X-Forwarded-For' % (self._x_forwarded_for_ip, country.upper()))
|
||||||
'[debug] Using fake IP %s (%s) as X-Forwarded-For.'
|
|
||||||
% (self._x_forwarded_for_ip, country.upper()))
|
|
||||||
|
|
||||||
def extract(self, url):
|
def extract(self, url):
|
||||||
"""Extracts URL information and returns it in list of dicts."""
|
"""Extracts URL information and returns it in list of dicts."""
|
||||||
@@ -554,12 +560,15 @@ class InfoExtractor(object):
|
|||||||
for _ in range(2):
|
for _ in range(2):
|
||||||
try:
|
try:
|
||||||
self.initialize()
|
self.initialize()
|
||||||
|
self.write_debug('Extracting URL: %s' % url)
|
||||||
ie_result = self._real_extract(url)
|
ie_result = self._real_extract(url)
|
||||||
|
if ie_result is None:
|
||||||
|
return None
|
||||||
if self._x_forwarded_for_ip:
|
if self._x_forwarded_for_ip:
|
||||||
ie_result['__x_forwarded_for_ip'] = self._x_forwarded_for_ip
|
ie_result['__x_forwarded_for_ip'] = self._x_forwarded_for_ip
|
||||||
subtitles = ie_result.get('subtitles')
|
subtitles = ie_result.get('subtitles')
|
||||||
if (subtitles and 'live_chat' in subtitles
|
if (subtitles and 'live_chat' in subtitles
|
||||||
and 'no-live-chat' in self._downloader.params.get('compat_opts')):
|
and 'no-live-chat' in self.get_param('compat_opts', [])):
|
||||||
del subtitles['live_chat']
|
del subtitles['live_chat']
|
||||||
return ie_result
|
return ie_result
|
||||||
except GeoRestrictedError as e:
|
except GeoRestrictedError as e:
|
||||||
@@ -574,9 +583,9 @@ class InfoExtractor(object):
|
|||||||
raise ExtractorError('An extractor error has occurred.', cause=e)
|
raise ExtractorError('An extractor error has occurred.', cause=e)
|
||||||
|
|
||||||
def __maybe_fake_ip_and_retry(self, countries):
|
def __maybe_fake_ip_and_retry(self, countries):
|
||||||
if (not self._downloader.params.get('geo_bypass_country', None)
|
if (not self.get_param('geo_bypass_country', None)
|
||||||
and self._GEO_BYPASS
|
and self._GEO_BYPASS
|
||||||
and self._downloader.params.get('geo_bypass', True)
|
and self.get_param('geo_bypass', True)
|
||||||
and not self._x_forwarded_for_ip
|
and not self._x_forwarded_for_ip
|
||||||
and countries):
|
and countries):
|
||||||
country_code = random.choice(countries)
|
country_code = random.choice(countries)
|
||||||
@@ -630,7 +639,7 @@ class InfoExtractor(object):
|
|||||||
See _download_webpage docstring for arguments specification.
|
See _download_webpage docstring for arguments specification.
|
||||||
"""
|
"""
|
||||||
if not self._downloader._first_webpage_request:
|
if not self._downloader._first_webpage_request:
|
||||||
sleep_interval = float_or_none(self._downloader.params.get('sleep_interval_requests')) or 0
|
sleep_interval = float_or_none(self.get_param('sleep_interval_requests')) or 0
|
||||||
if sleep_interval > 0:
|
if sleep_interval > 0:
|
||||||
self.to_screen('Sleeping %s seconds ...' % sleep_interval)
|
self.to_screen('Sleeping %s seconds ...' % sleep_interval)
|
||||||
time.sleep(sleep_interval)
|
time.sleep(sleep_interval)
|
||||||
@@ -755,11 +764,11 @@ class InfoExtractor(object):
|
|||||||
webpage_bytes = prefix + webpage_bytes
|
webpage_bytes = prefix + webpage_bytes
|
||||||
if not encoding:
|
if not encoding:
|
||||||
encoding = self._guess_encoding_from_content(content_type, webpage_bytes)
|
encoding = self._guess_encoding_from_content(content_type, webpage_bytes)
|
||||||
if self._downloader.params.get('dump_intermediate_pages', False):
|
if self.get_param('dump_intermediate_pages', False):
|
||||||
self.to_screen('Dumping request to ' + urlh.geturl())
|
self.to_screen('Dumping request to ' + urlh.geturl())
|
||||||
dump = base64.b64encode(webpage_bytes).decode('ascii')
|
dump = base64.b64encode(webpage_bytes).decode('ascii')
|
||||||
self._downloader.to_screen(dump)
|
self._downloader.to_screen(dump)
|
||||||
if self._downloader.params.get('write_pages', False):
|
if self.get_param('write_pages', False):
|
||||||
basen = '%s_%s' % (video_id, urlh.geturl())
|
basen = '%s_%s' % (video_id, urlh.geturl())
|
||||||
if len(basen) > 240:
|
if len(basen) > 240:
|
||||||
h = '___' + hashlib.md5(basen.encode('utf-8')).hexdigest()
|
h = '___' + hashlib.md5(basen.encode('utf-8')).hexdigest()
|
||||||
@@ -943,14 +952,22 @@ class InfoExtractor(object):
|
|||||||
else:
|
else:
|
||||||
self.report_warning(errmsg + str(ve))
|
self.report_warning(errmsg + str(ve))
|
||||||
|
|
||||||
def report_warning(self, msg, video_id=None):
|
def report_warning(self, msg, video_id=None, *args, **kwargs):
|
||||||
idstr = '' if video_id is None else '%s: ' % video_id
|
idstr = '' if video_id is None else '%s: ' % video_id
|
||||||
self._downloader.report_warning(
|
self._downloader.report_warning(
|
||||||
'[%s] %s%s' % (self.IE_NAME, idstr, msg))
|
'[%s] %s%s' % (self.IE_NAME, idstr, msg), *args, **kwargs)
|
||||||
|
|
||||||
def to_screen(self, msg):
|
def to_screen(self, msg, *args, **kwargs):
|
||||||
"""Print msg to screen, prefixing it with '[ie_name]'"""
|
"""Print msg to screen, prefixing it with '[ie_name]'"""
|
||||||
self._downloader.to_screen('[%s] %s' % (self.IE_NAME, msg))
|
self._downloader.to_screen('[%s] %s' % (self.IE_NAME, msg), *args, **kwargs)
|
||||||
|
|
||||||
|
def write_debug(self, msg, *args, **kwargs):
|
||||||
|
self._downloader.write_debug('[%s] %s' % (self.IE_NAME, msg), *args, **kwargs)
|
||||||
|
|
||||||
|
def get_param(self, name, default=None, *args, **kwargs):
|
||||||
|
if self._downloader:
|
||||||
|
return self._downloader.params.get(name, default, *args, **kwargs)
|
||||||
|
return default
|
||||||
|
|
||||||
def report_extraction(self, id_or_name):
|
def report_extraction(self, id_or_name):
|
||||||
"""Report information extraction."""
|
"""Report information extraction."""
|
||||||
@@ -969,23 +986,22 @@ class InfoExtractor(object):
|
|||||||
self.to_screen('Logging in')
|
self.to_screen('Logging in')
|
||||||
|
|
||||||
def raise_login_required(
|
def raise_login_required(
|
||||||
self, msg='This video is only available for registered users', metadata_available=False):
|
self, msg='This video is only available for registered users',
|
||||||
if metadata_available and self._downloader.params.get('ignore_no_formats_error'):
|
metadata_available=False, method='any'):
|
||||||
|
if metadata_available and self.get_param('ignore_no_formats_error'):
|
||||||
self.report_warning(msg)
|
self.report_warning(msg)
|
||||||
raise ExtractorError(
|
raise ExtractorError('%s. %s' % (msg, self._LOGIN_HINTS[method]), expected=True)
|
||||||
'%s. Use --cookies, --username and --password or --netrc to provide account credentials' % msg,
|
|
||||||
expected=True)
|
|
||||||
|
|
||||||
def raise_geo_restricted(
|
def raise_geo_restricted(
|
||||||
self, msg='This video is not available from your location due to geo restriction',
|
self, msg='This video is not available from your location due to geo restriction',
|
||||||
countries=None, metadata_available=False):
|
countries=None, metadata_available=False):
|
||||||
if metadata_available and self._downloader.params.get('ignore_no_formats_error'):
|
if metadata_available and self.get_param('ignore_no_formats_error'):
|
||||||
self.report_warning(msg)
|
self.report_warning(msg)
|
||||||
else:
|
else:
|
||||||
raise GeoRestrictedError(msg, countries=countries)
|
raise GeoRestrictedError(msg, countries=countries)
|
||||||
|
|
||||||
def raise_no_formats(self, msg, expected=False, video_id=None):
|
def raise_no_formats(self, msg, expected=False, video_id=None):
|
||||||
if expected and self._downloader.params.get('ignore_no_formats_error'):
|
if expected and self.get_param('ignore_no_formats_error'):
|
||||||
self.report_warning(msg, video_id)
|
self.report_warning(msg, video_id)
|
||||||
else:
|
else:
|
||||||
raise ExtractorError(msg, expected=expected, video_id=video_id)
|
raise ExtractorError(msg, expected=expected, video_id=video_id)
|
||||||
@@ -1040,7 +1056,7 @@ class InfoExtractor(object):
|
|||||||
if mobj:
|
if mobj:
|
||||||
break
|
break
|
||||||
|
|
||||||
if not self._downloader.params.get('no_color') and compat_os_name != 'nt' and sys.stderr.isatty():
|
if not self.get_param('no_color') and compat_os_name != 'nt' and sys.stderr.isatty():
|
||||||
_name = '\033[0;34m%s\033[0m' % name
|
_name = '\033[0;34m%s\033[0m' % name
|
||||||
else:
|
else:
|
||||||
_name = name
|
_name = name
|
||||||
@@ -1074,7 +1090,7 @@ class InfoExtractor(object):
|
|||||||
password = None
|
password = None
|
||||||
netrc_machine = netrc_machine or self._NETRC_MACHINE
|
netrc_machine = netrc_machine or self._NETRC_MACHINE
|
||||||
|
|
||||||
if self._downloader.params.get('usenetrc', False):
|
if self.get_param('usenetrc', False):
|
||||||
try:
|
try:
|
||||||
info = netrc.netrc().authenticators(netrc_machine)
|
info = netrc.netrc().authenticators(netrc_machine)
|
||||||
if info is not None:
|
if info is not None:
|
||||||
@@ -1098,15 +1114,11 @@ class InfoExtractor(object):
|
|||||||
value.
|
value.
|
||||||
If there's no info available, return (None, None)
|
If there's no info available, return (None, None)
|
||||||
"""
|
"""
|
||||||
if self._downloader is None:
|
|
||||||
return (None, None)
|
|
||||||
|
|
||||||
downloader_params = self._downloader.params
|
|
||||||
|
|
||||||
# Attempt to use provided username and password or .netrc data
|
# Attempt to use provided username and password or .netrc data
|
||||||
if downloader_params.get(username_option) is not None:
|
username = self.get_param(username_option)
|
||||||
username = downloader_params[username_option]
|
if username is not None:
|
||||||
password = downloader_params[password_option]
|
password = self.get_param(password_option)
|
||||||
else:
|
else:
|
||||||
username, password = self._get_netrc_login_info(netrc_machine)
|
username, password = self._get_netrc_login_info(netrc_machine)
|
||||||
|
|
||||||
@@ -1119,12 +1131,10 @@ class InfoExtractor(object):
|
|||||||
currently just uses the command line option
|
currently just uses the command line option
|
||||||
If there's no info available, return None
|
If there's no info available, return None
|
||||||
"""
|
"""
|
||||||
if self._downloader is None:
|
|
||||||
return None
|
|
||||||
downloader_params = self._downloader.params
|
|
||||||
|
|
||||||
if downloader_params.get('twofactor') is not None:
|
tfa = self.get_param('twofactor')
|
||||||
return downloader_params['twofactor']
|
if tfa is not None:
|
||||||
|
return tfa
|
||||||
|
|
||||||
return compat_getpass('Type %s and press [Return]: ' % note)
|
return compat_getpass('Type %s and press [Return]: ' % note)
|
||||||
|
|
||||||
@@ -1594,12 +1604,12 @@ class InfoExtractor(object):
|
|||||||
else limits[0] if has_limit and not has_multiple_limits
|
else limits[0] if has_limit and not has_multiple_limits
|
||||||
else None)
|
else None)
|
||||||
|
|
||||||
def print_verbose_info(self, to_screen):
|
def print_verbose_info(self, write_debug):
|
||||||
if self._sort_user:
|
if self._sort_user:
|
||||||
to_screen('[debug] Sort order given by user: %s' % ', '.join(self._sort_user))
|
write_debug('Sort order given by user: %s' % ', '.join(self._sort_user))
|
||||||
if self._sort_extractor:
|
if self._sort_extractor:
|
||||||
to_screen('[debug] Sort order given by extractor: %s' % ', '.join(self._sort_extractor))
|
write_debug('Sort order given by extractor: %s' % ', '.join(self._sort_extractor))
|
||||||
to_screen('[debug] Formats sorted by: %s' % ', '.join(['%s%s%s' % (
|
write_debug('Formats sorted by: %s' % ', '.join(['%s%s%s' % (
|
||||||
'+' if self._get_field_setting(field, 'reverse') else '', field,
|
'+' if self._get_field_setting(field, 'reverse') else '', field,
|
||||||
'%s%s(%s)' % ('~' if self._get_field_setting(field, 'closest') else ':',
|
'%s%s(%s)' % ('~' if self._get_field_setting(field, 'closest') else ':',
|
||||||
self._get_field_setting(field, 'limit_text'),
|
self._get_field_setting(field, 'limit_text'),
|
||||||
@@ -1685,13 +1695,13 @@ class InfoExtractor(object):
|
|||||||
|
|
||||||
def _sort_formats(self, formats, field_preference=[]):
|
def _sort_formats(self, formats, field_preference=[]):
|
||||||
if not formats:
|
if not formats:
|
||||||
if self._downloader.params.get('ignore_no_formats_error'):
|
if self.get_param('ignore_no_formats_error'):
|
||||||
return
|
return
|
||||||
raise ExtractorError('No video formats found')
|
raise ExtractorError('No video formats found')
|
||||||
format_sort = self.FormatSort() # params and to_screen are taken from the downloader
|
format_sort = self.FormatSort() # params and to_screen are taken from the downloader
|
||||||
format_sort.evaluate_params(self._downloader.params, field_preference)
|
format_sort.evaluate_params(self._downloader.params, field_preference)
|
||||||
if self._downloader.params.get('verbose', False):
|
if self.get_param('verbose', False):
|
||||||
format_sort.print_verbose_info(self._downloader.to_screen)
|
format_sort.print_verbose_info(self._downloader.write_debug)
|
||||||
formats.sort(key=lambda f: format_sort.calculate_preference(f))
|
formats.sort(key=lambda f: format_sort.calculate_preference(f))
|
||||||
|
|
||||||
def _check_formats(self, formats, video_id):
|
def _check_formats(self, formats, video_id):
|
||||||
@@ -1730,7 +1740,7 @@ class InfoExtractor(object):
|
|||||||
""" Either "http:" or "https:", depending on the user's preferences """
|
""" Either "http:" or "https:", depending on the user's preferences """
|
||||||
return (
|
return (
|
||||||
'http:'
|
'http:'
|
||||||
if self._downloader.params.get('prefer_insecure', False)
|
if self.get_param('prefer_insecure', False)
|
||||||
else 'https:')
|
else 'https:')
|
||||||
|
|
||||||
def _proto_relative_url(self, url, scheme=None):
|
def _proto_relative_url(self, url, scheme=None):
|
||||||
@@ -1924,7 +1934,7 @@ class InfoExtractor(object):
|
|||||||
if '#EXT-X-FAXS-CM:' in m3u8_doc: # Adobe Flash Access
|
if '#EXT-X-FAXS-CM:' in m3u8_doc: # Adobe Flash Access
|
||||||
return [], {}
|
return [], {}
|
||||||
|
|
||||||
if (not self._downloader.params.get('allow_unplayable_formats')
|
if (not self.get_param('allow_unplayable_formats')
|
||||||
and re.search(r'#EXT-X-SESSION-KEY:.*?URI="skd://', m3u8_doc)): # Apple FairPlay
|
and re.search(r'#EXT-X-SESSION-KEY:.*?URI="skd://', m3u8_doc)): # Apple FairPlay
|
||||||
return [], {}
|
return [], {}
|
||||||
|
|
||||||
@@ -1937,7 +1947,7 @@ class InfoExtractor(object):
|
|||||||
if re.match(r'^https?://', u)
|
if re.match(r'^https?://', u)
|
||||||
else compat_urlparse.urljoin(m3u8_url, u))
|
else compat_urlparse.urljoin(m3u8_url, u))
|
||||||
|
|
||||||
split_discontinuity = self._downloader.params.get('hls_split_discontinuity', False)
|
split_discontinuity = self.get_param('hls_split_discontinuity', False)
|
||||||
|
|
||||||
# References:
|
# References:
|
||||||
# 1. https://tools.ietf.org/html/draft-pantos-http-live-streaming-21
|
# 1. https://tools.ietf.org/html/draft-pantos-http-live-streaming-21
|
||||||
@@ -2480,7 +2490,7 @@ class InfoExtractor(object):
|
|||||||
http://standards.iso.org/ittf/PubliclyAvailableStandards/c065274_ISO_IEC_23009-1_2014.zip
|
http://standards.iso.org/ittf/PubliclyAvailableStandards/c065274_ISO_IEC_23009-1_2014.zip
|
||||||
2. https://en.wikipedia.org/wiki/Dynamic_Adaptive_Streaming_over_HTTP
|
2. https://en.wikipedia.org/wiki/Dynamic_Adaptive_Streaming_over_HTTP
|
||||||
"""
|
"""
|
||||||
if not self._downloader.params.get('dynamic_mpd', True):
|
if not self.get_param('dynamic_mpd', True):
|
||||||
if mpd_doc.get('type') == 'dynamic':
|
if mpd_doc.get('type') == 'dynamic':
|
||||||
return [], {}
|
return [], {}
|
||||||
|
|
||||||
@@ -2550,7 +2560,7 @@ class InfoExtractor(object):
|
|||||||
extract_Initialization(segment_template)
|
extract_Initialization(segment_template)
|
||||||
return ms_info
|
return ms_info
|
||||||
|
|
||||||
skip_unplayable = not self._downloader.params.get('allow_unplayable_formats')
|
skip_unplayable = not self.get_param('allow_unplayable_formats')
|
||||||
|
|
||||||
mpd_duration = parse_duration(mpd_doc.get('mediaPresentationDuration'))
|
mpd_duration = parse_duration(mpd_doc.get('mediaPresentationDuration'))
|
||||||
formats = []
|
formats = []
|
||||||
@@ -2799,7 +2809,7 @@ class InfoExtractor(object):
|
|||||||
"""
|
"""
|
||||||
if ism_doc.get('IsLive') == 'TRUE':
|
if ism_doc.get('IsLive') == 'TRUE':
|
||||||
return [], {}
|
return [], {}
|
||||||
if (not self._downloader.params.get('allow_unplayable_formats')
|
if (not self.get_param('allow_unplayable_formats')
|
||||||
and ism_doc.find('Protection') is not None):
|
and ism_doc.find('Protection') is not None):
|
||||||
return [], {}
|
return [], {}
|
||||||
|
|
||||||
@@ -3404,8 +3414,8 @@ class InfoExtractor(object):
|
|||||||
return not any_restricted
|
return not any_restricted
|
||||||
|
|
||||||
def extract_subtitles(self, *args, **kwargs):
|
def extract_subtitles(self, *args, **kwargs):
|
||||||
if (self._downloader.params.get('writesubtitles', False)
|
if (self.get_param('writesubtitles', False)
|
||||||
or self._downloader.params.get('listsubtitles')):
|
or self.get_param('listsubtitles')):
|
||||||
return self._get_subtitles(*args, **kwargs)
|
return self._get_subtitles(*args, **kwargs)
|
||||||
return {}
|
return {}
|
||||||
|
|
||||||
@@ -3440,8 +3450,8 @@ class InfoExtractor(object):
|
|||||||
return target
|
return target
|
||||||
|
|
||||||
def extract_automatic_captions(self, *args, **kwargs):
|
def extract_automatic_captions(self, *args, **kwargs):
|
||||||
if (self._downloader.params.get('writeautomaticsub', False)
|
if (self.get_param('writeautomaticsub', False)
|
||||||
or self._downloader.params.get('listsubtitles')):
|
or self.get_param('listsubtitles')):
|
||||||
return self._get_automatic_captions(*args, **kwargs)
|
return self._get_automatic_captions(*args, **kwargs)
|
||||||
return {}
|
return {}
|
||||||
|
|
||||||
@@ -3449,9 +3459,9 @@ class InfoExtractor(object):
|
|||||||
raise NotImplementedError('This method must be implemented by subclasses')
|
raise NotImplementedError('This method must be implemented by subclasses')
|
||||||
|
|
||||||
def mark_watched(self, *args, **kwargs):
|
def mark_watched(self, *args, **kwargs):
|
||||||
if (self._downloader.params.get('mark_watched', False)
|
if (self.get_param('mark_watched', False)
|
||||||
and (self._get_login_info()[0] is not None
|
and (self._get_login_info()[0] is not None
|
||||||
or self._downloader.params.get('cookiefile') is not None)):
|
or self.get_param('cookiefile') is not None)):
|
||||||
self._mark_watched(*args, **kwargs)
|
self._mark_watched(*args, **kwargs)
|
||||||
|
|
||||||
def _mark_watched(self, *args, **kwargs):
|
def _mark_watched(self, *args, **kwargs):
|
||||||
@@ -3459,7 +3469,7 @@ class InfoExtractor(object):
|
|||||||
|
|
||||||
def geo_verification_headers(self):
|
def geo_verification_headers(self):
|
||||||
headers = {}
|
headers = {}
|
||||||
geo_verification_proxy = self._downloader.params.get('geo_verification_proxy')
|
geo_verification_proxy = self.get_param('geo_verification_proxy')
|
||||||
if geo_verification_proxy:
|
if geo_verification_proxy:
|
||||||
headers['Ytdl-request-proxy'] = geo_verification_proxy
|
headers['Ytdl-request-proxy'] = geo_verification_proxy
|
||||||
return headers
|
return headers
|
||||||
|
|||||||
@@ -26,8 +26,8 @@ class CommonMistakesIE(InfoExtractor):
|
|||||||
'That doesn\'t make any sense. '
|
'That doesn\'t make any sense. '
|
||||||
'Simply remove the parameter in your command or configuration.'
|
'Simply remove the parameter in your command or configuration.'
|
||||||
) % url
|
) % url
|
||||||
if not self._downloader.params.get('verbose'):
|
if not self.get_param('verbose'):
|
||||||
msg += ' Add -v to the command line to see what arguments and configuration yt-dlp got.'
|
msg += ' Add -v to the command line to see what arguments and configuration yt-dlp has'
|
||||||
raise ExtractorError(msg, expected=True)
|
raise ExtractorError(msg, expected=True)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -81,7 +81,7 @@ class CrackleIE(InfoExtractor):
|
|||||||
def _real_extract(self, url):
|
def _real_extract(self, url):
|
||||||
video_id = self._match_id(url)
|
video_id = self._match_id(url)
|
||||||
|
|
||||||
geo_bypass_country = self._downloader.params.get('geo_bypass_country', None)
|
geo_bypass_country = self.get_param('geo_bypass_country', None)
|
||||||
countries = orderedSet((geo_bypass_country, 'US', 'AU', 'CA', 'AS', 'FM', 'GU', 'MP', 'PR', 'PW', 'MH', 'VI', ''))
|
countries = orderedSet((geo_bypass_country, 'US', 'AU', 'CA', 'AS', 'FM', 'GU', 'MP', 'PR', 'PW', 'MH', 'VI', ''))
|
||||||
num_countries, num = len(countries) - 1, 0
|
num_countries, num = len(countries) - 1, 0
|
||||||
|
|
||||||
@@ -128,8 +128,8 @@ class CrackleIE(InfoExtractor):
|
|||||||
if isinstance(media.get('MediaURLs'), list):
|
if isinstance(media.get('MediaURLs'), list):
|
||||||
break
|
break
|
||||||
|
|
||||||
ignore_no_formats = self._downloader.params.get('ignore_no_formats_error')
|
ignore_no_formats = self.get_param('ignore_no_formats_error')
|
||||||
allow_unplayable_formats = self._downloader.params.get('allow_unplayable_formats')
|
allow_unplayable_formats = self.get_param('allow_unplayable_formats')
|
||||||
|
|
||||||
if not media or (not media.get('MediaURLs') and not ignore_no_formats):
|
if not media or (not media.get('MediaURLs') and not ignore_no_formats):
|
||||||
raise ExtractorError(
|
raise ExtractorError(
|
||||||
|
|||||||
@@ -42,7 +42,7 @@ class DailymotionBaseInfoExtractor(InfoExtractor):
|
|||||||
def _real_initialize(self):
|
def _real_initialize(self):
|
||||||
cookies = self._get_dailymotion_cookies()
|
cookies = self._get_dailymotion_cookies()
|
||||||
ff = self._get_cookie_value(cookies, 'ff')
|
ff = self._get_cookie_value(cookies, 'ff')
|
||||||
self._FAMILY_FILTER = ff == 'on' if ff else age_restricted(18, self._downloader.params.get('age_limit'))
|
self._FAMILY_FILTER = ff == 'on' if ff else age_restricted(18, self.get_param('age_limit'))
|
||||||
self._set_dailymotion_cookie('ff', 'on' if self._FAMILY_FILTER else 'off')
|
self._set_dailymotion_cookie('ff', 'on' if self._FAMILY_FILTER else 'off')
|
||||||
|
|
||||||
def _call_api(self, object_type, xid, object_fields, note, filter_extra=None):
|
def _call_api(self, object_type, xid, object_fields, note, filter_extra=None):
|
||||||
@@ -207,14 +207,14 @@ class DailymotionIE(DailymotionBaseInfoExtractor):
|
|||||||
video_id, playlist_id = re.match(self._VALID_URL, url).groups()
|
video_id, playlist_id = re.match(self._VALID_URL, url).groups()
|
||||||
|
|
||||||
if playlist_id:
|
if playlist_id:
|
||||||
if not self._downloader.params.get('noplaylist'):
|
if not self.get_param('noplaylist'):
|
||||||
self.to_screen('Downloading playlist %s - add --no-playlist to just download video' % playlist_id)
|
self.to_screen('Downloading playlist %s - add --no-playlist to just download video' % playlist_id)
|
||||||
return self.url_result(
|
return self.url_result(
|
||||||
'http://www.dailymotion.com/playlist/' + playlist_id,
|
'http://www.dailymotion.com/playlist/' + playlist_id,
|
||||||
'DailymotionPlaylist', playlist_id)
|
'DailymotionPlaylist', playlist_id)
|
||||||
self.to_screen('Downloading just video %s because of --no-playlist' % video_id)
|
self.to_screen('Downloading just video %s because of --no-playlist' % video_id)
|
||||||
|
|
||||||
password = self._downloader.params.get('videopassword')
|
password = self.get_param('videopassword')
|
||||||
media = self._call_api(
|
media = self._call_api(
|
||||||
'media', video_id, '''... on Video {
|
'media', video_id, '''... on Video {
|
||||||
%s
|
%s
|
||||||
@@ -232,7 +232,7 @@ class DailymotionIE(DailymotionBaseInfoExtractor):
|
|||||||
audienceCount
|
audienceCount
|
||||||
isOnAir
|
isOnAir
|
||||||
}''' % (self._COMMON_MEDIA_FIELDS, self._COMMON_MEDIA_FIELDS), 'Downloading media JSON metadata',
|
}''' % (self._COMMON_MEDIA_FIELDS, self._COMMON_MEDIA_FIELDS), 'Downloading media JSON metadata',
|
||||||
'password: "%s"' % self._downloader.params.get('videopassword') if password else None)
|
'password: "%s"' % self.get_param('videopassword') if password else None)
|
||||||
xid = media['xid']
|
xid = media['xid']
|
||||||
|
|
||||||
metadata = self._download_json(
|
metadata = self._download_json(
|
||||||
|
|||||||
@@ -158,7 +158,7 @@ class DaumListIE(InfoExtractor):
|
|||||||
query_dict = compat_parse_qs(compat_urlparse.urlparse(url).query)
|
query_dict = compat_parse_qs(compat_urlparse.urlparse(url).query)
|
||||||
if 'clipid' in query_dict:
|
if 'clipid' in query_dict:
|
||||||
clip_id = query_dict['clipid'][0]
|
clip_id = query_dict['clipid'][0]
|
||||||
if self._downloader.params.get('noplaylist'):
|
if self.get_param('noplaylist'):
|
||||||
self.to_screen('Downloading just video %s because of --no-playlist' % clip_id)
|
self.to_screen('Downloading just video %s because of --no-playlist' % clip_id)
|
||||||
return self.url_result(DaumClipIE._URL_TEMPLATE % clip_id, 'DaumClip')
|
return self.url_result(DaumClipIE._URL_TEMPLATE % clip_id, 'DaumClip')
|
||||||
else:
|
else:
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ from ..utils import (
|
|||||||
|
|
||||||
class DeezerBaseInfoExtractor(InfoExtractor):
|
class DeezerBaseInfoExtractor(InfoExtractor):
|
||||||
def get_data(self, url):
|
def get_data(self, url):
|
||||||
if not self._downloader.params.get('test'):
|
if not self.get_param('test'):
|
||||||
self.report_warning('For now, this extractor only supports the 30 second previews. Patches welcome!')
|
self.report_warning('For now, this extractor only supports the 30 second previews. Patches welcome!')
|
||||||
|
|
||||||
mobj = re.match(self._VALID_URL, url)
|
mobj = re.match(self._VALID_URL, url)
|
||||||
|
|||||||
@@ -6,7 +6,7 @@ from .common import InfoExtractor
|
|||||||
from ..compat import compat_urllib_parse_urlencode
|
from ..compat import compat_urllib_parse_urlencode
|
||||||
from ..utils import (
|
from ..utils import (
|
||||||
ExtractorError,
|
ExtractorError,
|
||||||
unescapeHTML
|
merge_dicts,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -24,7 +24,8 @@ class EroProfileIE(InfoExtractor):
|
|||||||
'title': 'sexy babe softcore',
|
'title': 'sexy babe softcore',
|
||||||
'thumbnail': r're:https?://.*\.jpg',
|
'thumbnail': r're:https?://.*\.jpg',
|
||||||
'age_limit': 18,
|
'age_limit': 18,
|
||||||
}
|
},
|
||||||
|
'skip': 'Video not found',
|
||||||
}, {
|
}, {
|
||||||
'url': 'http://www.eroprofile.com/m/videos/view/Try-It-On-Pee_cut_2-wmv-4shared-com-file-sharing-download-movie-file',
|
'url': 'http://www.eroprofile.com/m/videos/view/Try-It-On-Pee_cut_2-wmv-4shared-com-file-sharing-download-movie-file',
|
||||||
'md5': '1baa9602ede46ce904c431f5418d8916',
|
'md5': '1baa9602ede46ce904c431f5418d8916',
|
||||||
@@ -77,19 +78,15 @@ class EroProfileIE(InfoExtractor):
|
|||||||
[r"glbUpdViews\s*\('\d*','(\d+)'", r'p/report/video/(\d+)'],
|
[r"glbUpdViews\s*\('\d*','(\d+)'", r'p/report/video/(\d+)'],
|
||||||
webpage, 'video id', default=None)
|
webpage, 'video id', default=None)
|
||||||
|
|
||||||
video_url = unescapeHTML(self._search_regex(
|
|
||||||
r'<source src="([^"]+)', webpage, 'video url'))
|
|
||||||
title = self._html_search_regex(
|
title = self._html_search_regex(
|
||||||
r'Title:</th><td>([^<]+)</td>', webpage, 'title')
|
(r'Title:</th><td>([^<]+)</td>', r'<h1[^>]*>(.+?)</h1>'),
|
||||||
thumbnail = self._search_regex(
|
webpage, 'title')
|
||||||
r'onclick="showVideoPlayer\(\)"><img src="([^"]+)',
|
|
||||||
webpage, 'thumbnail', fatal=False)
|
|
||||||
|
|
||||||
return {
|
info = self._parse_html5_media_entries(url, webpage, video_id)[0]
|
||||||
|
|
||||||
|
return merge_dicts(info, {
|
||||||
'id': video_id,
|
'id': video_id,
|
||||||
'display_id': display_id,
|
'display_id': display_id,
|
||||||
'url': video_url,
|
|
||||||
'title': title,
|
'title': title,
|
||||||
'thumbnail': thumbnail,
|
|
||||||
'age_limit': 18,
|
'age_limit': 18,
|
||||||
}
|
})
|
||||||
|
|||||||
@@ -94,7 +94,8 @@ from .audiomack import AudiomackIE, AudiomackAlbumIE
|
|||||||
from .audius import (
|
from .audius import (
|
||||||
AudiusIE,
|
AudiusIE,
|
||||||
AudiusTrackIE,
|
AudiusTrackIE,
|
||||||
AudiusPlaylistIE
|
AudiusPlaylistIE,
|
||||||
|
AudiusProfileIE,
|
||||||
)
|
)
|
||||||
from .awaan import (
|
from .awaan import (
|
||||||
AWAANIE,
|
AWAANIE,
|
||||||
@@ -952,6 +953,7 @@ from .palcomp3 import (
|
|||||||
)
|
)
|
||||||
from .pandoratv import PandoraTVIE
|
from .pandoratv import PandoraTVIE
|
||||||
from .parliamentliveuk import ParliamentLiveUKIE
|
from .parliamentliveuk import ParliamentLiveUKIE
|
||||||
|
from .parlview import ParlviewIE
|
||||||
from .patreon import PatreonIE
|
from .patreon import PatreonIE
|
||||||
from .pbs import PBSIE
|
from .pbs import PBSIE
|
||||||
from .pearvideo import PearVideoIE
|
from .pearvideo import PearVideoIE
|
||||||
@@ -983,6 +985,7 @@ from .platzi import (
|
|||||||
from .playfm import PlayFMIE
|
from .playfm import PlayFMIE
|
||||||
from .playplustv import PlayPlusTVIE
|
from .playplustv import PlayPlusTVIE
|
||||||
from .plays import PlaysTVIE
|
from .plays import PlaysTVIE
|
||||||
|
from .playstuff import PlayStuffIE
|
||||||
from .playtvak import PlaytvakIE
|
from .playtvak import PlaytvakIE
|
||||||
from .playvid import PlayvidIE
|
from .playvid import PlayvidIE
|
||||||
from .playwire import PlaywireIE
|
from .playwire import PlaywireIE
|
||||||
|
|||||||
@@ -126,6 +126,7 @@ from .viqeo import ViqeoIE
|
|||||||
from .expressen import ExpressenIE
|
from .expressen import ExpressenIE
|
||||||
from .zype import ZypeIE
|
from .zype import ZypeIE
|
||||||
from .odnoklassniki import OdnoklassnikiIE
|
from .odnoklassniki import OdnoklassnikiIE
|
||||||
|
from .vk import VKIE
|
||||||
from .kinja import KinjaEmbedIE
|
from .kinja import KinjaEmbedIE
|
||||||
from .gedidigital import GediDigitalIE
|
from .gedidigital import GediDigitalIE
|
||||||
from .rcs import RCSEmbedsIE
|
from .rcs import RCSEmbedsIE
|
||||||
@@ -2252,6 +2253,10 @@ class GenericIE(InfoExtractor):
|
|||||||
'playlist_mincount': 52,
|
'playlist_mincount': 52,
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
|
# Sibnet embed (https://help.sibnet.ru/?sibnet_video_embed)
|
||||||
|
'url': 'https://phpbb3.x-tk.ru/bbcode-video-sibnet-t24.html',
|
||||||
|
'only_matching': True,
|
||||||
|
}, {
|
||||||
# WimTv embed player
|
# WimTv embed player
|
||||||
'url': 'http://www.msmotor.tv/wearefmi-pt-2-2021/',
|
'url': 'http://www.msmotor.tv/wearefmi-pt-2-2021/',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
@@ -2370,7 +2375,7 @@ class GenericIE(InfoExtractor):
|
|||||||
|
|
||||||
parsed_url = compat_urlparse.urlparse(url)
|
parsed_url = compat_urlparse.urlparse(url)
|
||||||
if not parsed_url.scheme:
|
if not parsed_url.scheme:
|
||||||
default_search = self._downloader.params.get('default_search')
|
default_search = self.get_param('default_search')
|
||||||
if default_search is None:
|
if default_search is None:
|
||||||
default_search = 'fixup_error'
|
default_search = 'fixup_error'
|
||||||
|
|
||||||
@@ -2461,8 +2466,8 @@ class GenericIE(InfoExtractor):
|
|||||||
info_dict['subtitles'] = subtitles
|
info_dict['subtitles'] = subtitles
|
||||||
return info_dict
|
return info_dict
|
||||||
|
|
||||||
if not self._downloader.params.get('test', False) and not is_intentional:
|
if not self.get_param('test', False) and not is_intentional:
|
||||||
force = self._downloader.params.get('force_generic_extractor', False)
|
force = self.get_param('force_generic_extractor', False)
|
||||||
self.report_warning(
|
self.report_warning(
|
||||||
'%s on generic information extractor.' % ('Forcing' if force else 'Falling back'))
|
'%s on generic information extractor.' % ('Forcing' if force else 'Falling back'))
|
||||||
|
|
||||||
@@ -2803,6 +2808,11 @@ class GenericIE(InfoExtractor):
|
|||||||
if odnoklassniki_url:
|
if odnoklassniki_url:
|
||||||
return self.url_result(odnoklassniki_url, OdnoklassnikiIE.ie_key())
|
return self.url_result(odnoklassniki_url, OdnoklassnikiIE.ie_key())
|
||||||
|
|
||||||
|
# Look for sibnet embedded player
|
||||||
|
sibnet_urls = VKIE._extract_sibnet_urls(webpage)
|
||||||
|
if sibnet_urls:
|
||||||
|
return self.playlist_from_matches(sibnet_urls, video_id, video_title)
|
||||||
|
|
||||||
# Look for embedded ivi player
|
# Look for embedded ivi player
|
||||||
mobj = re.search(r'<embed[^>]+?src=(["\'])(?P<url>https?://(?:www\.)?ivi\.ru/video/player.+?)\1', webpage)
|
mobj = re.search(r'<embed[^>]+?src=(["\'])(?P<url>https?://(?:www\.)?ivi\.ru/video/player.+?)\1', webpage)
|
||||||
if mobj is not None:
|
if mobj is not None:
|
||||||
@@ -3454,6 +3464,9 @@ class GenericIE(InfoExtractor):
|
|||||||
'url': src,
|
'url': src,
|
||||||
'ext': (mimetype2ext(src_type)
|
'ext': (mimetype2ext(src_type)
|
||||||
or ext if ext in KNOWN_EXTENSIONS else 'mp4'),
|
or ext if ext in KNOWN_EXTENSIONS else 'mp4'),
|
||||||
|
'http_headers': {
|
||||||
|
'Referer': full_response.geturl(),
|
||||||
|
},
|
||||||
})
|
})
|
||||||
if formats:
|
if formats:
|
||||||
self._sort_formats(formats)
|
self._sort_formats(formats)
|
||||||
@@ -3522,7 +3535,7 @@ class GenericIE(InfoExtractor):
|
|||||||
m_video_type = re.findall(r'<meta.*?property="og:video:type".*?content="video/(.*?)"', webpage)
|
m_video_type = re.findall(r'<meta.*?property="og:video:type".*?content="video/(.*?)"', webpage)
|
||||||
# We only look in og:video if the MIME type is a video, don't try if it's a Flash player:
|
# We only look in og:video if the MIME type is a video, don't try if it's a Flash player:
|
||||||
if m_video_type is not None:
|
if m_video_type is not None:
|
||||||
found = filter_video(re.findall(r'<meta.*?property="og:video".*?content="(.*?)"', webpage))
|
found = filter_video(re.findall(r'<meta.*?property="og:(?:video|audio)".*?content="(.*?)"', webpage))
|
||||||
if not found:
|
if not found:
|
||||||
REDIRECT_REGEX = r'[0-9]{,2};\s*(?:URL|url)=\'?([^\'"]+)'
|
REDIRECT_REGEX = r'[0-9]{,2};\s*(?:URL|url)=\'?([^\'"]+)'
|
||||||
found = re.search(
|
found = re.search(
|
||||||
|
|||||||
@@ -96,7 +96,7 @@ class GloboIE(InfoExtractor):
|
|||||||
video = self._download_json(
|
video = self._download_json(
|
||||||
'http://api.globovideos.com/videos/%s/playlist' % video_id,
|
'http://api.globovideos.com/videos/%s/playlist' % video_id,
|
||||||
video_id)['videos'][0]
|
video_id)['videos'][0]
|
||||||
if not self._downloader.params.get('allow_unplayable_formats') and video.get('encrypted') is True:
|
if not self.get_param('allow_unplayable_formats') and video.get('encrypted') is True:
|
||||||
raise ExtractorError('This video is DRM protected.', expected=True)
|
raise ExtractorError('This video is DRM protected.', expected=True)
|
||||||
|
|
||||||
title = video['title']
|
title = video['title']
|
||||||
|
|||||||
@@ -141,7 +141,7 @@ class HotStarIE(HotStarBaseIE):
|
|||||||
|
|
||||||
title = video_data['title']
|
title = video_data['title']
|
||||||
|
|
||||||
if not self._downloader.params.get('allow_unplayable_formats') and video_data.get('drmProtected'):
|
if not self.get_param('allow_unplayable_formats') and video_data.get('drmProtected'):
|
||||||
raise ExtractorError('This video is DRM protected.', expected=True)
|
raise ExtractorError('This video is DRM protected.', expected=True)
|
||||||
|
|
||||||
headers = {'Referer': url}
|
headers = {'Referer': url}
|
||||||
|
|||||||
@@ -65,7 +65,7 @@ class ImgGamingBaseIE(InfoExtractor):
|
|||||||
domain, media_type, media_id, playlist_id = re.match(self._VALID_URL, url).groups()
|
domain, media_type, media_id, playlist_id = re.match(self._VALID_URL, url).groups()
|
||||||
|
|
||||||
if playlist_id:
|
if playlist_id:
|
||||||
if self._downloader.params.get('noplaylist'):
|
if self.get_param('noplaylist'):
|
||||||
self.to_screen('Downloading just video %s because of --no-playlist' % media_id)
|
self.to_screen('Downloading just video %s because of --no-playlist' % media_id)
|
||||||
else:
|
else:
|
||||||
self.to_screen('Downloading playlist %s - add --no-playlist to just download video' % playlist_id)
|
self.to_screen('Downloading playlist %s - add --no-playlist to just download video' % playlist_id)
|
||||||
|
|||||||
@@ -165,7 +165,7 @@ class IviIE(InfoExtractor):
|
|||||||
content_format = f.get('content_format')
|
content_format = f.get('content_format')
|
||||||
if not f_url:
|
if not f_url:
|
||||||
continue
|
continue
|
||||||
if (not self._downloader.params.get('allow_unplayable_formats')
|
if (not self.get_param('allow_unplayable_formats')
|
||||||
and ('-MDRM-' in content_format or '-FPS-' in content_format)):
|
and ('-MDRM-' in content_format or '-FPS-' in content_format)):
|
||||||
continue
|
continue
|
||||||
formats.append({
|
formats.append({
|
||||||
|
|||||||
@@ -309,7 +309,7 @@ class KalturaIE(InfoExtractor):
|
|||||||
if f.get('fileExt') == 'chun':
|
if f.get('fileExt') == 'chun':
|
||||||
continue
|
continue
|
||||||
# DRM-protected video, cannot be decrypted
|
# DRM-protected video, cannot be decrypted
|
||||||
if not self._downloader.params.get('allow_unplayable_formats') and f.get('fileExt') == 'wvm':
|
if not self.get_param('allow_unplayable_formats') and f.get('fileExt') == 'wvm':
|
||||||
continue
|
continue
|
||||||
if not f.get('fileExt'):
|
if not f.get('fileExt'):
|
||||||
# QT indicates QuickTime; some videos have broken fileExt
|
# QT indicates QuickTime; some videos have broken fileExt
|
||||||
|
|||||||
@@ -98,7 +98,7 @@ class LimelightBaseIE(InfoExtractor):
|
|||||||
stream_url = stream.get('url')
|
stream_url = stream.get('url')
|
||||||
if not stream_url or stream_url in urls:
|
if not stream_url or stream_url in urls:
|
||||||
continue
|
continue
|
||||||
if not self._downloader.params.get('allow_unplayable_formats') and stream.get('drmProtected'):
|
if not self.get_param('allow_unplayable_formats') and stream.get('drmProtected'):
|
||||||
continue
|
continue
|
||||||
urls.append(stream_url)
|
urls.append(stream_url)
|
||||||
ext = determine_ext(stream_url)
|
ext = determine_ext(stream_url)
|
||||||
@@ -163,7 +163,7 @@ class LimelightBaseIE(InfoExtractor):
|
|||||||
if not media_url or media_url in urls:
|
if not media_url or media_url in urls:
|
||||||
continue
|
continue
|
||||||
if (format_id in ('Widevine', 'SmoothStreaming')
|
if (format_id in ('Widevine', 'SmoothStreaming')
|
||||||
and not self._downloader.params.get('allow_unplayable_formats', False)):
|
and not self.get_param('allow_unplayable_formats', False)):
|
||||||
continue
|
continue
|
||||||
urls.append(media_url)
|
urls.append(media_url)
|
||||||
ext = determine_ext(media_url)
|
ext = determine_ext(media_url)
|
||||||
|
|||||||
@@ -71,7 +71,7 @@ class LiTVIE(InfoExtractor):
|
|||||||
|
|
||||||
video_id = self._match_id(url)
|
video_id = self._match_id(url)
|
||||||
|
|
||||||
noplaylist = self._downloader.params.get('noplaylist')
|
noplaylist = self.get_param('noplaylist')
|
||||||
noplaylist_prompt = True
|
noplaylist_prompt = True
|
||||||
if 'force_noplaylist' in data:
|
if 'force_noplaylist' in data:
|
||||||
noplaylist = data['force_noplaylist']
|
noplaylist = data['force_noplaylist']
|
||||||
|
|||||||
@@ -167,7 +167,7 @@ class NBAWatchIE(NBAWatchBaseIE):
|
|||||||
display_id = self._match_id(url)
|
display_id = self._match_id(url)
|
||||||
collection_id = compat_parse_qs(compat_urllib_parse_urlparse(url).query).get('collection', [None])[0]
|
collection_id = compat_parse_qs(compat_urllib_parse_urlparse(url).query).get('collection', [None])[0]
|
||||||
if collection_id:
|
if collection_id:
|
||||||
if self._downloader.params.get('noplaylist'):
|
if self.get_param('noplaylist'):
|
||||||
self.to_screen('Downloading just video %s because of --no-playlist' % display_id)
|
self.to_screen('Downloading just video %s because of --no-playlist' % display_id)
|
||||||
else:
|
else:
|
||||||
self.to_screen('Downloading playlist %s - add --no-playlist to just download video' % collection_id)
|
self.to_screen('Downloading playlist %s - add --no-playlist to just download video' % collection_id)
|
||||||
|
|||||||
@@ -405,7 +405,7 @@ class NetEaseMusicProgramIE(NetEaseMusicBaseIE):
|
|||||||
name = info['name']
|
name = info['name']
|
||||||
description = info['description']
|
description = info['description']
|
||||||
|
|
||||||
if not info['songs'] or self._downloader.params.get('noplaylist'):
|
if not info['songs'] or self.get_param('noplaylist'):
|
||||||
if info['songs']:
|
if info['songs']:
|
||||||
self.to_screen(
|
self.to_screen(
|
||||||
'Downloading just the main audio %s because of --no-playlist'
|
'Downloading just the main audio %s because of --no-playlist'
|
||||||
|
|||||||
@@ -34,7 +34,7 @@ class NineCNineMediaIE(InfoExtractor):
|
|||||||
'$include': '[HasClosedCaptions]',
|
'$include': '[HasClosedCaptions]',
|
||||||
})
|
})
|
||||||
|
|
||||||
if (not self._downloader.params.get('allow_unplayable_formats')
|
if (not self.get_param('allow_unplayable_formats')
|
||||||
and try_get(content_package, lambda x: x['Constraints']['Security']['Type'])):
|
and try_get(content_package, lambda x: x['Constraints']['Security']['Type'])):
|
||||||
raise ExtractorError('This video is DRM protected.', expected=True)
|
raise ExtractorError('This video is DRM protected.', expected=True)
|
||||||
|
|
||||||
|
|||||||
@@ -66,7 +66,7 @@ class NineNowIE(InfoExtractor):
|
|||||||
|
|
||||||
video_data = common_data['video']
|
video_data = common_data['video']
|
||||||
|
|
||||||
if not self._downloader.params.get('allow_unplayable_formats') and video_data.get('drm'):
|
if not self.get_param('allow_unplayable_formats') and video_data.get('drm'):
|
||||||
raise ExtractorError('This video is DRM protected.', expected=True)
|
raise ExtractorError('This video is DRM protected.', expected=True)
|
||||||
|
|
||||||
brightcove_id = video_data.get('brightcoveId') or 'ref:' + video_data['referenceId']
|
brightcove_id = video_data.get('brightcoveId') or 'ref:' + video_data['referenceId']
|
||||||
|
|||||||
@@ -246,7 +246,7 @@ class NPOIE(NPOBaseIE):
|
|||||||
})
|
})
|
||||||
|
|
||||||
if not formats:
|
if not formats:
|
||||||
if not self._downloader.params.get('allow_unplayable_formats') and drm:
|
if not self.get_param('allow_unplayable_formats') and drm:
|
||||||
self.raise_no_formats('This video is DRM protected.', expected=True)
|
self.raise_no_formats('This video is DRM protected.', expected=True)
|
||||||
return
|
return
|
||||||
|
|
||||||
|
|||||||
@@ -182,7 +182,7 @@ class OnetChannelIE(OnetBaseIE):
|
|||||||
video_id = remove_start(current_clip_info['ckmId'], 'mvp:')
|
video_id = remove_start(current_clip_info['ckmId'], 'mvp:')
|
||||||
video_name = url_basename(current_clip_info['url'])
|
video_name = url_basename(current_clip_info['url'])
|
||||||
|
|
||||||
if self._downloader.params.get('noplaylist'):
|
if self.get_param('noplaylist'):
|
||||||
self.to_screen(
|
self.to_screen(
|
||||||
'Downloading just video %s because of --no-playlist' % video_name)
|
'Downloading just video %s because of --no-playlist' % video_name)
|
||||||
return self._extract_from_id(video_id, webpage)
|
return self._extract_from_id(video_id, webpage)
|
||||||
|
|||||||
@@ -182,7 +182,7 @@ class ORFRadioIE(InfoExtractor):
|
|||||||
duration = end - start if end and start else None
|
duration = end - start if end and start else None
|
||||||
entries.append({
|
entries.append({
|
||||||
'id': loop_stream_id.replace('.mp3', ''),
|
'id': loop_stream_id.replace('.mp3', ''),
|
||||||
'url': 'http://loopstream01.apa.at/?channel=%s&id=%s' % (self._LOOP_STATION, loop_stream_id),
|
'url': 'https://loopstream01.apa.at/?channel=%s&id=%s' % (self._LOOP_STATION, loop_stream_id),
|
||||||
'title': title,
|
'title': title,
|
||||||
'description': clean_html(data.get('subtitle')),
|
'description': clean_html(data.get('subtitle')),
|
||||||
'duration': duration,
|
'duration': duration,
|
||||||
|
|||||||
@@ -0,0 +1,68 @@
|
|||||||
|
# coding: utf-8
|
||||||
|
from __future__ import unicode_literals
|
||||||
|
|
||||||
|
from .common import InfoExtractor
|
||||||
|
from ..compat import compat_str
|
||||||
|
from ..utils import (
|
||||||
|
int_or_none,
|
||||||
|
try_get,
|
||||||
|
unified_timestamp,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class ParlviewIE(InfoExtractor):
|
||||||
|
|
||||||
|
_VALID_URL = r'https?://(?:www\.)?parlview\.aph\.gov\.au/(?:[^/]+)?\bvideoID=(?P<id>\d{6})'
|
||||||
|
_TESTS = [{
|
||||||
|
'url': 'https://parlview.aph.gov.au/mediaPlayer.php?videoID=542661',
|
||||||
|
'info_dict': {
|
||||||
|
'id': '542661',
|
||||||
|
'ext': 'mp4',
|
||||||
|
'title': "Australia's Family Law System [Part 2]",
|
||||||
|
'duration': 5799,
|
||||||
|
'description': 'md5:7099883b391619dbae435891ca871a62',
|
||||||
|
'timestamp': 1621430700,
|
||||||
|
'upload_date': '20210519',
|
||||||
|
'uploader': 'Joint Committee',
|
||||||
|
},
|
||||||
|
'params': {
|
||||||
|
'skip_download': True,
|
||||||
|
}
|
||||||
|
}, {
|
||||||
|
'url': 'https://parlview.aph.gov.au/mediaPlayer.php?videoID=539936',
|
||||||
|
'only_matching': True,
|
||||||
|
}]
|
||||||
|
_API_URL = 'https://parlview.aph.gov.au/api_v3/1/playback/getUniversalPlayerConfig?videoID=%s&format=json'
|
||||||
|
_MEDIA_INFO_URL = 'https://parlview.aph.gov.au/ajaxPlayer.php?videoID=%s&tabNum=4&action=loadTab'
|
||||||
|
|
||||||
|
def _real_extract(self, url):
|
||||||
|
video_id = self._match_id(url)
|
||||||
|
webpage = self._download_webpage(url, video_id)
|
||||||
|
media = self._download_json(self._API_URL % video_id, video_id).get('media')
|
||||||
|
timestamp = try_get(media, lambda x: x['timeMap']['source']['timecode_offsets'][0], compat_str) or '/'
|
||||||
|
|
||||||
|
stream = try_get(media, lambda x: x['renditions'][0], dict)
|
||||||
|
if not stream:
|
||||||
|
self.raise_no_formats('No streams were detected')
|
||||||
|
elif stream.get('streamType') != 'VOD':
|
||||||
|
self.raise_no_formats('Unknown type of stream was detected: "%s"' % str(stream.get('streamType')))
|
||||||
|
formats = self._extract_m3u8_formats(stream['url'], video_id, 'mp4', 'm3u8_native')
|
||||||
|
self._sort_formats(formats)
|
||||||
|
|
||||||
|
media_info = self._download_webpage(
|
||||||
|
self._MEDIA_INFO_URL % video_id, video_id, note='Downloading media info', fatal=False)
|
||||||
|
|
||||||
|
return {
|
||||||
|
'id': video_id,
|
||||||
|
'url': url,
|
||||||
|
'title': self._html_search_regex(r'<h2>([^<]+)<', webpage, 'title', fatal=False),
|
||||||
|
'formats': formats,
|
||||||
|
'duration': int_or_none(media.get('duration')),
|
||||||
|
'timestamp': unified_timestamp(timestamp.split('/', 1)[1].replace('_', ' ')),
|
||||||
|
'description': self._html_search_regex(
|
||||||
|
r'<div[^>]+class="descripti?on"[^>]*>[^>]+<strong>[^>]+>[^>]+>([^<]+)',
|
||||||
|
webpage, 'description', fatal=False),
|
||||||
|
'uploader': self._html_search_regex(
|
||||||
|
r'<td>[^>]+>Channel:[^>]+>([^<]+)', media_info, 'channel', fatal=False),
|
||||||
|
'thumbnail': media.get('staticImage'),
|
||||||
|
}
|
||||||
@@ -79,7 +79,7 @@ class PhilharmonieDeParisIE(InfoExtractor):
|
|||||||
formats.extend(self._extract_m3u8_formats(
|
formats.extend(self._extract_m3u8_formats(
|
||||||
m3u8_url, video_id, 'mp4', entry_protocol='m3u8_native',
|
m3u8_url, video_id, 'mp4', entry_protocol='m3u8_native',
|
||||||
m3u8_id='hls', fatal=False))
|
m3u8_id='hls', fatal=False))
|
||||||
if not formats and not self._downloader.params.get('ignore_no_formats'):
|
if not formats and not self.get_param('ignore_no_formats'):
|
||||||
return
|
return
|
||||||
self._sort_formats(formats)
|
self._sort_formats(formats)
|
||||||
return {
|
return {
|
||||||
|
|||||||
+28
-23
@@ -9,8 +9,9 @@ from ..compat import compat_str
|
|||||||
from ..utils import (
|
from ..utils import (
|
||||||
int_or_none,
|
int_or_none,
|
||||||
merge_dicts,
|
merge_dicts,
|
||||||
|
try_get,
|
||||||
unified_timestamp,
|
unified_timestamp,
|
||||||
xpath_text,
|
urljoin,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -27,10 +28,11 @@ class PhoenixIE(ZDFBaseIE):
|
|||||||
'title': 'Wohin führt der Protest in der Pandemie?',
|
'title': 'Wohin führt der Protest in der Pandemie?',
|
||||||
'description': 'md5:7d643fe7f565e53a24aac036b2122fbd',
|
'description': 'md5:7d643fe7f565e53a24aac036b2122fbd',
|
||||||
'duration': 1691,
|
'duration': 1691,
|
||||||
'timestamp': 1613906100,
|
'timestamp': 1613902500,
|
||||||
'upload_date': '20210221',
|
'upload_date': '20210221',
|
||||||
'uploader': 'Phoenix',
|
'uploader': 'Phoenix',
|
||||||
'channel': 'corona nachgehakt',
|
'series': 'corona nachgehakt',
|
||||||
|
'episode': 'Wohin führt der Protest in der Pandemie?',
|
||||||
},
|
},
|
||||||
}, {
|
}, {
|
||||||
# Youtube embed
|
# Youtube embed
|
||||||
@@ -79,50 +81,53 @@ class PhoenixIE(ZDFBaseIE):
|
|||||||
|
|
||||||
video_id = compat_str(video.get('basename') or video.get('content'))
|
video_id = compat_str(video.get('basename') or video.get('content'))
|
||||||
|
|
||||||
details = self._download_xml(
|
details = self._download_json(
|
||||||
'https://www.phoenix.de/php/mediaplayer/data/beitrags_details.php',
|
'https://www.phoenix.de/php/mediaplayer/data/beitrags_details.php',
|
||||||
video_id, 'Downloading details XML', query={
|
video_id, 'Downloading details JSON', query={
|
||||||
'ak': 'web',
|
'ak': 'web',
|
||||||
'ptmd': 'true',
|
'ptmd': 'true',
|
||||||
'id': video_id,
|
'id': video_id,
|
||||||
'profile': 'player2',
|
'profile': 'player2',
|
||||||
})
|
})
|
||||||
|
|
||||||
title = title or xpath_text(
|
title = title or details['title']
|
||||||
details, './/information/title', 'title', fatal=True)
|
content_id = details['tracking']['nielsen']['content']['assetid']
|
||||||
content_id = xpath_text(
|
|
||||||
details, './/video/details/basename', 'content id', fatal=True)
|
|
||||||
|
|
||||||
info = self._extract_ptmd(
|
info = self._extract_ptmd(
|
||||||
'https://tmd.phoenix.de/tmd/2/ngplayer_2_3/vod/ptmd/phoenix/%s' % content_id,
|
'https://tmd.phoenix.de/tmd/2/ngplayer_2_3/vod/ptmd/phoenix/%s' % content_id,
|
||||||
content_id, None, url)
|
content_id, None, url)
|
||||||
|
|
||||||
timestamp = unified_timestamp(xpath_text(details, './/details/airtime'))
|
duration = int_or_none(try_get(
|
||||||
|
details, lambda x: x['tracking']['nielsen']['content']['length']))
|
||||||
|
timestamp = unified_timestamp(details.get('editorialDate'))
|
||||||
|
series = try_get(
|
||||||
|
details, lambda x: x['tracking']['nielsen']['content']['program'],
|
||||||
|
compat_str)
|
||||||
|
episode = title if details.get('contentType') == 'episode' else None
|
||||||
|
|
||||||
thumbnails = []
|
thumbnails = []
|
||||||
for node in details.findall('.//teaserimages/teaserimage'):
|
teaser_images = try_get(details, lambda x: x['teaserImageRef']['layouts'], dict) or {}
|
||||||
thumbnail_url = node.text
|
for thumbnail_key, thumbnail_url in teaser_images.items():
|
||||||
|
thumbnail_url = urljoin(url, thumbnail_url)
|
||||||
if not thumbnail_url:
|
if not thumbnail_url:
|
||||||
continue
|
continue
|
||||||
thumbnail = {
|
thumbnail = {
|
||||||
'url': thumbnail_url,
|
'url': thumbnail_url,
|
||||||
}
|
}
|
||||||
thumbnail_key = node.get('key')
|
m = re.match('^([0-9]+)x([0-9]+)$', thumbnail_key)
|
||||||
if thumbnail_key:
|
if m:
|
||||||
m = re.match('^([0-9]+)x([0-9]+)$', thumbnail_key)
|
thumbnail['width'] = int(m.group(1))
|
||||||
if m:
|
thumbnail['height'] = int(m.group(2))
|
||||||
thumbnail['width'] = int(m.group(1))
|
|
||||||
thumbnail['height'] = int(m.group(2))
|
|
||||||
thumbnails.append(thumbnail)
|
thumbnails.append(thumbnail)
|
||||||
|
|
||||||
return merge_dicts(info, {
|
return merge_dicts(info, {
|
||||||
'id': content_id,
|
'id': content_id,
|
||||||
'title': title,
|
'title': title,
|
||||||
'description': xpath_text(details, './/information/detail'),
|
'description': details.get('leadParagraph'),
|
||||||
'duration': int_or_none(xpath_text(details, './/details/lengthSec')),
|
'duration': duration,
|
||||||
'thumbnails': thumbnails,
|
'thumbnails': thumbnails,
|
||||||
'timestamp': timestamp,
|
'timestamp': timestamp,
|
||||||
'uploader': xpath_text(details, './/details/channel'),
|
'uploader': details.get('tvService'),
|
||||||
'uploader_id': xpath_text(details, './/details/originChannelId'),
|
'series': series,
|
||||||
'channel': xpath_text(details, './/details/originChannelTitle'),
|
'episode': episode,
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -0,0 +1,65 @@
|
|||||||
|
from __future__ import unicode_literals
|
||||||
|
|
||||||
|
from .common import InfoExtractor
|
||||||
|
from ..compat import compat_str
|
||||||
|
from ..utils import (
|
||||||
|
smuggle_url,
|
||||||
|
try_get,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class PlayStuffIE(InfoExtractor):
|
||||||
|
_VALID_URL = r'https?://(?:www\.)?play\.stuff\.co\.nz/details/(?P<id>[^/?#&]+)'
|
||||||
|
_TESTS = [{
|
||||||
|
'url': 'https://play.stuff.co.nz/details/608778ac1de1c4001a3fa09a',
|
||||||
|
'md5': 'c82d3669e5247c64bc382577843e5bd0',
|
||||||
|
'info_dict': {
|
||||||
|
'id': '6250584958001',
|
||||||
|
'ext': 'mp4',
|
||||||
|
'title': 'Episode 1: Rotorua/Mt Maunganui/Tauranga',
|
||||||
|
'description': 'md5:c154bafb9f0dd02d01fd4100fb1c1913',
|
||||||
|
'uploader_id': '6005208634001',
|
||||||
|
'timestamp': 1619491027,
|
||||||
|
'upload_date': '20210427',
|
||||||
|
},
|
||||||
|
'add_ie': ['BrightcoveNew'],
|
||||||
|
}, {
|
||||||
|
# geo restricted, bypassable
|
||||||
|
'url': 'https://play.stuff.co.nz/details/_6155660351001',
|
||||||
|
'only_matching': True,
|
||||||
|
}]
|
||||||
|
BRIGHTCOVE_URL_TEMPLATE = 'http://players.brightcove.net/%s/%s_default/index.html?videoId=%s'
|
||||||
|
|
||||||
|
def _real_extract(self, url):
|
||||||
|
video_id = self._match_id(url)
|
||||||
|
|
||||||
|
webpage = self._download_webpage(url, video_id)
|
||||||
|
|
||||||
|
state = self._parse_json(
|
||||||
|
self._search_regex(
|
||||||
|
r'__INITIAL_STATE__\s*=\s*({.+?})\s*;', webpage, 'state'),
|
||||||
|
video_id)
|
||||||
|
|
||||||
|
account_id = try_get(
|
||||||
|
state, lambda x: x['configurations']['accountId'],
|
||||||
|
compat_str) or '6005208634001'
|
||||||
|
player_id = try_get(
|
||||||
|
state, lambda x: x['configurations']['playerId'],
|
||||||
|
compat_str) or 'default'
|
||||||
|
|
||||||
|
entries = []
|
||||||
|
for item_id, video in state['items'].items():
|
||||||
|
if not isinstance(video, dict):
|
||||||
|
continue
|
||||||
|
asset_id = try_get(
|
||||||
|
video, lambda x: x['content']['attributes']['assetId'],
|
||||||
|
compat_str)
|
||||||
|
if not asset_id:
|
||||||
|
continue
|
||||||
|
entries.append(self.url_result(
|
||||||
|
smuggle_url(
|
||||||
|
self.BRIGHTCOVE_URL_TEMPLATE % (account_id, player_id, asset_id),
|
||||||
|
{'geo_countries': ['NZ']}),
|
||||||
|
'BrightcoveNew', video_id))
|
||||||
|
|
||||||
|
return self.playlist_result(entries, video_id)
|
||||||
@@ -337,11 +337,11 @@ query viewClip {
|
|||||||
# In order to minimize the number of calls to ViewClip API and reduce
|
# In order to minimize the number of calls to ViewClip API and reduce
|
||||||
# the probability of being throttled or banned by Pluralsight we will request
|
# the probability of being throttled or banned by Pluralsight we will request
|
||||||
# only single format until formats listing was explicitly requested.
|
# only single format until formats listing was explicitly requested.
|
||||||
if self._downloader.params.get('listformats', False):
|
if self.get_param('listformats', False):
|
||||||
allowed_qualities = ALLOWED_QUALITIES
|
allowed_qualities = ALLOWED_QUALITIES
|
||||||
else:
|
else:
|
||||||
def guess_allowed_qualities():
|
def guess_allowed_qualities():
|
||||||
req_format = self._downloader.params.get('format') or 'best'
|
req_format = self.get_param('format') or 'best'
|
||||||
req_format_split = req_format.split('-', 1)
|
req_format_split = req_format.split('-', 1)
|
||||||
if len(req_format_split) > 1:
|
if len(req_format_split) > 1:
|
||||||
req_ext, req_quality = req_format_split
|
req_ext, req_quality = req_format_split
|
||||||
@@ -349,7 +349,7 @@ query viewClip {
|
|||||||
for allowed_quality in ALLOWED_QUALITIES:
|
for allowed_quality in ALLOWED_QUALITIES:
|
||||||
if req_ext == allowed_quality.ext and req_quality in allowed_quality.qualities:
|
if req_ext == allowed_quality.ext and req_quality in allowed_quality.qualities:
|
||||||
return (AllowedQuality(req_ext, (req_quality, )), )
|
return (AllowedQuality(req_ext, (req_quality, )), )
|
||||||
req_ext = 'webm' if self._downloader.params.get('prefer_free_formats') else 'mp4'
|
req_ext = 'webm' if self.get_param('prefer_free_formats') else 'mp4'
|
||||||
return (AllowedQuality(req_ext, (best_quality, )), )
|
return (AllowedQuality(req_ext, (best_quality, )), )
|
||||||
allowed_qualities = guess_allowed_qualities()
|
allowed_qualities = guess_allowed_qualities()
|
||||||
|
|
||||||
|
|||||||
@@ -109,7 +109,7 @@ class PlutoTVIE(InfoExtractor):
|
|||||||
if ad_free_formats:
|
if ad_free_formats:
|
||||||
formats, subtitles = ad_free_formats, ad_free_subtitles
|
formats, subtitles = ad_free_formats, ad_free_subtitles
|
||||||
else:
|
else:
|
||||||
self._downloader.report_warning('Unable to find ad-free formats')
|
self.report_warning('Unable to find ad-free formats')
|
||||||
return formats, subtitles
|
return formats, subtitles
|
||||||
|
|
||||||
def _get_video_info(self, video_json, slug, series_name=None):
|
def _get_video_info(self, video_json, slug, series_name=None):
|
||||||
|
|||||||
@@ -112,7 +112,7 @@ class PokemonWatchIE(InfoExtractor):
|
|||||||
}
|
}
|
||||||
|
|
||||||
# API call can be avoided entirely if we are listing formats
|
# API call can be avoided entirely if we are listing formats
|
||||||
if self._downloader.params.get('listformats', False):
|
if self.get_param('listformats', False):
|
||||||
return info
|
return info
|
||||||
|
|
||||||
webpage = self._download_webpage(url, video_id)
|
webpage = self._download_webpage(url, video_id)
|
||||||
|
|||||||
@@ -34,7 +34,7 @@ class ProSiebenSat1BaseIE(InfoExtractor):
|
|||||||
'ids': clip_id,
|
'ids': clip_id,
|
||||||
})[0]
|
})[0]
|
||||||
|
|
||||||
if not self._downloader.params.get('allow_unplayable_formats') and video.get('is_protected') is True:
|
if not self.get_param('allow_unplayable_formats') and video.get('is_protected') is True:
|
||||||
raise ExtractorError('This video is DRM protected.', expected=True)
|
raise ExtractorError('This video is DRM protected.', expected=True)
|
||||||
|
|
||||||
formats = []
|
formats = []
|
||||||
|
|||||||
@@ -275,7 +275,7 @@ class RaiPlayIE(RaiBaseIE):
|
|||||||
media = self._download_json(
|
media = self._download_json(
|
||||||
base + '.json', video_id, 'Downloading video JSON')
|
base + '.json', video_id, 'Downloading video JSON')
|
||||||
|
|
||||||
if not self._downloader.params.get('allow_unplayable_formats'):
|
if not self.get_param('allow_unplayable_formats'):
|
||||||
if try_get(
|
if try_get(
|
||||||
media,
|
media,
|
||||||
(lambda x: x['rights_management']['rights']['drm'],
|
(lambda x: x['rights_management']['rights']['drm'],
|
||||||
|
|||||||
@@ -133,8 +133,10 @@ class RedBullEmbedIE(RedBullTVIE):
|
|||||||
rrn_id = self._match_id(url)
|
rrn_id = self._match_id(url)
|
||||||
asset_id = self._download_json(
|
asset_id = self._download_json(
|
||||||
'https://edge-graphql.crepo-production.redbullaws.com/v1/graphql',
|
'https://edge-graphql.crepo-production.redbullaws.com/v1/graphql',
|
||||||
rrn_id, headers={'API-KEY': 'e90a1ff11335423998b100c929ecc866'},
|
rrn_id, headers={
|
||||||
query={
|
'Accept': 'application/json',
|
||||||
|
'API-KEY': 'e90a1ff11335423998b100c929ecc866',
|
||||||
|
}, query={
|
||||||
'query': '''{
|
'query': '''{
|
||||||
resource(id: "%s", enforceGeoBlocking: false) {
|
resource(id: "%s", enforceGeoBlocking: false) {
|
||||||
%s
|
%s
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ from ..utils import smuggle_url
|
|||||||
|
|
||||||
|
|
||||||
class RMCDecouverteIE(InfoExtractor):
|
class RMCDecouverteIE(InfoExtractor):
|
||||||
_VALID_URL = r'https?://rmcdecouverte\.bfmtv\.com/(?:[^/]+/(?P<id>[^?#/]+)|(?P<live_id>mediaplayer-direct))'
|
_VALID_URL = r'https?://rmcdecouverte\.bfmtv\.com/(?:[^?#]*_(?P<id>\d+)|mediaplayer-direct)/?(?:[#?]|$)'
|
||||||
|
|
||||||
_TESTS = [{
|
_TESTS = [{
|
||||||
'url': 'https://rmcdecouverte.bfmtv.com/vestiges-de-guerre_22240/les-bunkers-secrets-domaha-beach_25303/',
|
'url': 'https://rmcdecouverte.bfmtv.com/vestiges-de-guerre_22240/les-bunkers-secrets-domaha-beach_25303/',
|
||||||
@@ -45,6 +45,13 @@ class RMCDecouverteIE(InfoExtractor):
|
|||||||
'skip_download': True,
|
'skip_download': True,
|
||||||
},
|
},
|
||||||
'skip': 'only available for a week',
|
'skip': 'only available for a week',
|
||||||
|
}, {
|
||||||
|
'url': 'https://rmcdecouverte.bfmtv.com/avions-furtifs-la-technologie-de-lextreme_10598',
|
||||||
|
'only_matching': True,
|
||||||
|
}, {
|
||||||
|
# The website accepts any URL as long as it has _\d+ at the end
|
||||||
|
'url': 'https://rmcdecouverte.bfmtv.com/any/thing/can/go/here/_10598',
|
||||||
|
'only_matching': True,
|
||||||
}, {
|
}, {
|
||||||
# live, geo restricted, bypassable
|
# live, geo restricted, bypassable
|
||||||
'url': 'https://rmcdecouverte.bfmtv.com/mediaplayer-direct/',
|
'url': 'https://rmcdecouverte.bfmtv.com/mediaplayer-direct/',
|
||||||
@@ -54,7 +61,7 @@ class RMCDecouverteIE(InfoExtractor):
|
|||||||
|
|
||||||
def _real_extract(self, url):
|
def _real_extract(self, url):
|
||||||
mobj = re.match(self._VALID_URL, url)
|
mobj = re.match(self._VALID_URL, url)
|
||||||
display_id = mobj.group('id') or mobj.group('live_id')
|
display_id = mobj.group('id') or 'direct'
|
||||||
webpage = self._download_webpage(url, display_id)
|
webpage = self._download_webpage(url, display_id)
|
||||||
brightcove_legacy_url = BrightcoveLegacyIE._extract_brightcove_url(webpage)
|
brightcove_legacy_url = BrightcoveLegacyIE._extract_brightcove_url(webpage)
|
||||||
if brightcove_legacy_url:
|
if brightcove_legacy_url:
|
||||||
|
|||||||
@@ -125,7 +125,7 @@ class RTBFIE(InfoExtractor):
|
|||||||
})
|
})
|
||||||
|
|
||||||
mpd_url = data.get('urlDash')
|
mpd_url = data.get('urlDash')
|
||||||
if mpd_url and (self._downloader.params.get('allow_unplayable_formats') or not data.get('drm')):
|
if mpd_url and (self.get_param('allow_unplayable_formats') or not data.get('drm')):
|
||||||
formats.extend(self._extract_mpd_formats(
|
formats.extend(self._extract_mpd_formats(
|
||||||
mpd_url, media_id, mpd_id='dash', fatal=False))
|
mpd_url, media_id, mpd_id='dash', fatal=False))
|
||||||
|
|
||||||
|
|||||||
@@ -200,7 +200,7 @@ class RuutuIE(InfoExtractor):
|
|||||||
return node.get('value')
|
return node.get('value')
|
||||||
|
|
||||||
if not formats:
|
if not formats:
|
||||||
if (not self._downloader.params.get('allow_unplayable_formats')
|
if (not self.get_param('allow_unplayable_formats')
|
||||||
and xpath_text(video_xml, './Clip/DRM', default=None)):
|
and xpath_text(video_xml, './Clip/DRM', default=None)):
|
||||||
self.raise_no_formats('This video is DRM protected.', expected=True)
|
self.raise_no_formats('This video is DRM protected.', expected=True)
|
||||||
ns_st_cds = pv('ns_st_cds')
|
ns_st_cds = pv('ns_st_cds')
|
||||||
|
|||||||
@@ -21,6 +21,7 @@ from ..utils import (
|
|||||||
class ShahidBaseIE(AWSIE):
|
class ShahidBaseIE(AWSIE):
|
||||||
_AWS_PROXY_HOST = 'api2.shahid.net'
|
_AWS_PROXY_HOST = 'api2.shahid.net'
|
||||||
_AWS_API_KEY = '2RRtuMHx95aNI1Kvtn2rChEuwsCogUd4samGPjLh'
|
_AWS_API_KEY = '2RRtuMHx95aNI1Kvtn2rChEuwsCogUd4samGPjLh'
|
||||||
|
_VALID_URL_BASE = r'https?://shahid\.mbc\.net/[a-z]{2}/'
|
||||||
|
|
||||||
def _handle_error(self, e):
|
def _handle_error(self, e):
|
||||||
fail_data = self._parse_json(
|
fail_data = self._parse_json(
|
||||||
@@ -49,7 +50,7 @@ class ShahidBaseIE(AWSIE):
|
|||||||
|
|
||||||
class ShahidIE(ShahidBaseIE):
|
class ShahidIE(ShahidBaseIE):
|
||||||
_NETRC_MACHINE = 'shahid'
|
_NETRC_MACHINE = 'shahid'
|
||||||
_VALID_URL = r'https?://shahid\.mbc\.net/ar/(?:serie|show|movie)s/[^/]+/(?P<type>episode|clip|movie)-(?P<id>\d+)'
|
_VALID_URL = ShahidBaseIE._VALID_URL_BASE + r'(?:serie|show|movie)s/[^/]+/(?P<type>episode|clip|movie)-(?P<id>\d+)'
|
||||||
_TESTS = [{
|
_TESTS = [{
|
||||||
'url': 'https://shahid.mbc.net/ar/shows/%D9%85%D8%AA%D8%AD%D9%81-%D8%A7%D9%84%D8%AF%D8%AD%D9%8A%D8%AD-%D8%A7%D9%84%D9%85%D9%88%D8%B3%D9%85-1-%D9%83%D9%84%D9%8A%D8%A8-1/clip-816924',
|
'url': 'https://shahid.mbc.net/ar/shows/%D9%85%D8%AA%D8%AD%D9%81-%D8%A7%D9%84%D8%AF%D8%AD%D9%8A%D8%AD-%D8%A7%D9%84%D9%85%D9%88%D8%B3%D9%85-1-%D9%83%D9%84%D9%8A%D8%A8-1/clip-816924',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
@@ -73,6 +74,9 @@ class ShahidIE(ShahidBaseIE):
|
|||||||
# shahid plus subscriber only
|
# shahid plus subscriber only
|
||||||
'url': 'https://shahid.mbc.net/ar/series/%D9%85%D8%B1%D8%A7%D9%8A%D8%A7-2011-%D8%A7%D9%84%D9%85%D9%88%D8%B3%D9%85-1-%D8%A7%D9%84%D8%AD%D9%84%D9%82%D8%A9-1/episode-90511',
|
'url': 'https://shahid.mbc.net/ar/series/%D9%85%D8%B1%D8%A7%D9%8A%D8%A7-2011-%D8%A7%D9%84%D9%85%D9%88%D8%B3%D9%85-1-%D8%A7%D9%84%D8%AD%D9%84%D9%82%D8%A9-1/episode-90511',
|
||||||
'only_matching': True
|
'only_matching': True
|
||||||
|
}, {
|
||||||
|
'url': 'https://shahid.mbc.net/en/shows/Ramez-Fi-Al-Shallal-season-1-episode-1/episode-359319',
|
||||||
|
'only_matching': True
|
||||||
}]
|
}]
|
||||||
|
|
||||||
def _real_initialize(self):
|
def _real_initialize(self):
|
||||||
@@ -114,7 +118,7 @@ class ShahidIE(ShahidBaseIE):
|
|||||||
playout = self._call_api(
|
playout = self._call_api(
|
||||||
'playout/new/url/' + video_id, video_id)['playout']
|
'playout/new/url/' + video_id, video_id)['playout']
|
||||||
|
|
||||||
if not self._downloader.params.get('allow_unplayable_formats') and playout.get('drm'):
|
if not self.get_param('allow_unplayable_formats') and playout.get('drm'):
|
||||||
raise ExtractorError('This video is DRM protected.', expected=True)
|
raise ExtractorError('This video is DRM protected.', expected=True)
|
||||||
|
|
||||||
formats = self._extract_m3u8_formats(re.sub(
|
formats = self._extract_m3u8_formats(re.sub(
|
||||||
@@ -168,7 +172,7 @@ class ShahidIE(ShahidBaseIE):
|
|||||||
|
|
||||||
|
|
||||||
class ShahidShowIE(ShahidBaseIE):
|
class ShahidShowIE(ShahidBaseIE):
|
||||||
_VALID_URL = r'https?://shahid\.mbc\.net/ar/(?:show|serie)s/[^/]+/(?:show|series)-(?P<id>\d+)'
|
_VALID_URL = ShahidBaseIE._VALID_URL_BASE + r'(?:show|serie)s/[^/]+/(?:show|series)-(?P<id>\d+)'
|
||||||
_TESTS = [{
|
_TESTS = [{
|
||||||
'url': 'https://shahid.mbc.net/ar/shows/%D8%B1%D8%A7%D9%85%D8%B2-%D9%82%D8%B1%D8%B4-%D8%A7%D9%84%D8%A8%D8%AD%D8%B1/show-79187',
|
'url': 'https://shahid.mbc.net/ar/shows/%D8%B1%D8%A7%D9%85%D8%B2-%D9%82%D8%B1%D8%B4-%D8%A7%D9%84%D8%A8%D8%AD%D8%B1/show-79187',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
|
|||||||
@@ -86,10 +86,10 @@ class SharedIE(SharedBaseIE):
|
|||||||
|
|
||||||
class VivoIE(SharedBaseIE):
|
class VivoIE(SharedBaseIE):
|
||||||
IE_DESC = 'vivo.sx'
|
IE_DESC = 'vivo.sx'
|
||||||
_VALID_URL = r'https?://vivo\.sx/(?P<id>[\da-z]{10})'
|
_VALID_URL = r'https?://vivo\.s[xt]/(?P<id>[\da-z]{10})'
|
||||||
_FILE_NOT_FOUND = '>The file you have requested does not exists or has been removed'
|
_FILE_NOT_FOUND = '>The file you have requested does not exists or has been removed'
|
||||||
|
|
||||||
_TEST = {
|
_TESTS = [{
|
||||||
'url': 'http://vivo.sx/d7ddda0e78',
|
'url': 'http://vivo.sx/d7ddda0e78',
|
||||||
'md5': '15b3af41be0b4fe01f4df075c2678b2c',
|
'md5': '15b3af41be0b4fe01f4df075c2678b2c',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
@@ -98,7 +98,10 @@ class VivoIE(SharedBaseIE):
|
|||||||
'title': 'Chicken',
|
'title': 'Chicken',
|
||||||
'filesize': 515659,
|
'filesize': 515659,
|
||||||
},
|
},
|
||||||
}
|
}, {
|
||||||
|
'url': 'http://vivo.st/d7ddda0e78',
|
||||||
|
'only_matching': True,
|
||||||
|
}]
|
||||||
|
|
||||||
def _extract_title(self, webpage):
|
def _extract_title(self, webpage):
|
||||||
title = self._html_search_regex(
|
title = self._html_search_regex(
|
||||||
|
|||||||
@@ -75,7 +75,7 @@ class SonyLIVIE(InfoExtractor):
|
|||||||
video_id = self._match_id(url)
|
video_id = self._match_id(url)
|
||||||
content = self._call_api(
|
content = self._call_api(
|
||||||
'1.5', 'IN/CONTENT/VIDEOURL/VOD/' + video_id, video_id)
|
'1.5', 'IN/CONTENT/VIDEOURL/VOD/' + video_id, video_id)
|
||||||
if not self._downloader.params.get('allow_unplayable_formats') and content.get('isEncrypted'):
|
if not self.get_param('allow_unplayable_formats') and content.get('isEncrypted'):
|
||||||
raise ExtractorError('This video is DRM protected.', expected=True)
|
raise ExtractorError('This video is DRM protected.', expected=True)
|
||||||
dash_url = content['videoURL']
|
dash_url = content['videoURL']
|
||||||
headers = {
|
headers = {
|
||||||
|
|||||||
@@ -63,7 +63,7 @@ class SportDeutschlandIE(InfoExtractor):
|
|||||||
if len(videos) > 1:
|
if len(videos) > 1:
|
||||||
playlist_id = compat_parse_qs(compat_urllib_parse_urlparse(url).query).get('playlistId', [None])[0]
|
playlist_id = compat_parse_qs(compat_urllib_parse_urlparse(url).query).get('playlistId', [None])[0]
|
||||||
if playlist_id:
|
if playlist_id:
|
||||||
if self._downloader.params.get('noplaylist'):
|
if self.get_param('noplaylist'):
|
||||||
videos = [videos[int(playlist_id)]]
|
videos = [videos[int(playlist_id)]]
|
||||||
self.to_screen('Downloading just a single video because of --no-playlist')
|
self.to_screen('Downloading just a single video because of --no-playlist')
|
||||||
else:
|
else:
|
||||||
@@ -77,7 +77,7 @@ class SportDeutschlandIE(InfoExtractor):
|
|||||||
continue
|
continue
|
||||||
formats = self._extract_m3u8_formats(
|
formats = self._extract_m3u8_formats(
|
||||||
video_url.replace('.smil', '.m3u8'), video_id, 'mp4', fatal=False)
|
video_url.replace('.smil', '.m3u8'), video_id, 'mp4', fatal=False)
|
||||||
if not formats and not self._downloader.params.get('ignore_no_formats'):
|
if not formats and not self.get_param('ignore_no_formats'):
|
||||||
continue
|
continue
|
||||||
yield {
|
yield {
|
||||||
'id': video_id,
|
'id': video_id,
|
||||||
|
|||||||
@@ -139,7 +139,7 @@ class SteamIE(InfoExtractor):
|
|||||||
'format_id': ext + quality,
|
'format_id': ext + quality,
|
||||||
'url': video_url,
|
'url': video_url,
|
||||||
})
|
})
|
||||||
if not formats and not self._downloader.params.get('ignore_no_formats'):
|
if not formats and not self.get_param('ignore_no_formats'):
|
||||||
continue
|
continue
|
||||||
entry['formats'] = formats
|
entry['formats'] = formats
|
||||||
entries.append(entry)
|
entries.append(entry)
|
||||||
|
|||||||
+53
-34
@@ -1,70 +1,89 @@
|
|||||||
# coding: utf-8
|
# coding: utf-8
|
||||||
from __future__ import unicode_literals
|
from __future__ import unicode_literals
|
||||||
|
|
||||||
|
from datetime import datetime
|
||||||
|
import base64
|
||||||
|
|
||||||
from .common import InfoExtractor
|
from .common import InfoExtractor
|
||||||
from ..utils import (
|
from ..utils import (
|
||||||
HEADRequest,
|
HEADRequest,
|
||||||
parse_age_limit,
|
urlencode_postdata,
|
||||||
parse_iso8601,
|
|
||||||
# smuggle_url,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
class TenPlayIE(InfoExtractor):
|
class TenPlayIE(InfoExtractor):
|
||||||
_VALID_URL = r'https?://(?:www\.)?10play\.com\.au/(?:[^/]+/)+(?P<id>tpv\d{6}[a-z]{5})'
|
_VALID_URL = r'https?://(?:www\.)?10play\.com\.au/(?:[^/]+/)+(?P<id>tpv\d{6}[a-z]{5})'
|
||||||
|
_NETRC_MACHINE = '10play'
|
||||||
_TESTS = [{
|
_TESTS = [{
|
||||||
'url': 'https://10play.com.au/masterchef/episodes/season-1/masterchef-s1-ep-1/tpv190718kwzga',
|
'url': 'https://10play.com.au/todd-sampsons-body-hack/episodes/season-4/episode-7/tpv200921kvngh',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': '6060533435001',
|
'id': '6192880312001',
|
||||||
'ext': 'mp4',
|
'ext': 'mp4',
|
||||||
'title': 'MasterChef - S1 Ep. 1',
|
'title': "Todd Sampson's Body Hack - S4 Ep. 2",
|
||||||
'description': 'md5:4fe7b78e28af8f2d900cd20d900ef95c',
|
'description': 'md5:fa278820ad90f08ea187f9458316ac74',
|
||||||
'age_limit': 10,
|
'age_limit': 15,
|
||||||
'timestamp': 1240828200,
|
'timestamp': 1600770600,
|
||||||
'upload_date': '20090427',
|
'upload_date': '20200922',
|
||||||
'uploader_id': '2199827728001',
|
'uploader': 'Channel 10',
|
||||||
|
'uploader_id': '2199827728001'
|
||||||
},
|
},
|
||||||
'params': {
|
'params': {
|
||||||
# 'format': 'bestvideo',
|
|
||||||
'skip_download': True,
|
'skip_download': True,
|
||||||
}
|
}
|
||||||
}, {
|
}, {
|
||||||
'url': 'https://10play.com.au/how-to-stay-married/web-extras/season-1/terrys-talks-ep-1-embracing-change/tpv190915ylupc',
|
'url': 'https://10play.com.au/how-to-stay-married/web-extras/season-1/terrys-talks-ep-1-embracing-change/tpv190915ylupc',
|
||||||
'only_matching': True,
|
'only_matching': True,
|
||||||
}]
|
}]
|
||||||
# BRIGHTCOVE_URL_TEMPLATE = 'https://players.brightcove.net/2199827728001/cN6vRtRQt_default/index.html?videoId=%s'
|
|
||||||
_GEO_BYPASS = False
|
_GEO_BYPASS = False
|
||||||
_FASTLY_URL_TEMPL = 'https://10-selector.global.ssl.fastly.net/s/kYEXFC/media/%s?mbr=true&manifest=m3u&format=redirect'
|
|
||||||
|
_AUS_AGES = {
|
||||||
|
'G': 0,
|
||||||
|
'PG': 15,
|
||||||
|
'M': 15,
|
||||||
|
'MA': 15,
|
||||||
|
'R': 18,
|
||||||
|
'X': 18
|
||||||
|
}
|
||||||
|
|
||||||
|
def _get_bearer_token(self, video_id):
|
||||||
|
username, password = self._get_login_info()
|
||||||
|
if username is None or password is None:
|
||||||
|
self.raise_login_required('Your 10play account\'s details must be provided with --username and --password.')
|
||||||
|
_timestamp = datetime.now().strftime('%Y%m%d000000')
|
||||||
|
_auth_header = base64.b64encode(_timestamp.encode('ascii')).decode('ascii')
|
||||||
|
data = self._download_json('https://10play.com.au/api/user/auth', video_id, 'Getting bearer token', headers={
|
||||||
|
'X-Network-Ten-Auth': _auth_header,
|
||||||
|
}, data=urlencode_postdata({
|
||||||
|
'email': username,
|
||||||
|
'password': password,
|
||||||
|
}))
|
||||||
|
return "Bearer " + data['jwt']['accessToken']
|
||||||
|
|
||||||
def _real_extract(self, url):
|
def _real_extract(self, url):
|
||||||
content_id = self._match_id(url)
|
content_id = self._match_id(url)
|
||||||
|
_token = self._get_bearer_token(content_id)
|
||||||
data = self._download_json(
|
data = self._download_json(
|
||||||
'https://10play.com.au/api/video/' + content_id, content_id)
|
'https://10play.com.au/api/v1/videos/' + content_id, content_id)
|
||||||
video = data.get('video') or {}
|
_video_url = self._download_json(
|
||||||
metadata = data.get('metaData') or {}
|
data.get('playbackApiEndpoint'), content_id, 'Downloading video JSON',
|
||||||
brightcove_id = video.get('videoId') or metadata['showContentVideoId']
|
headers={'Authorization': _token}).get('source')
|
||||||
# brightcove_url = smuggle_url(
|
|
||||||
# self.BRIGHTCOVE_URL_TEMPLATE % brightcove_id,
|
|
||||||
# {'geo_countries': ['AU']})
|
|
||||||
m3u8_url = self._request_webpage(HEADRequest(
|
m3u8_url = self._request_webpage(HEADRequest(
|
||||||
self._FASTLY_URL_TEMPL % brightcove_id), brightcove_id).geturl()
|
_video_url), content_id).geturl()
|
||||||
if '10play-not-in-oz' in m3u8_url:
|
if '10play-not-in-oz' in m3u8_url:
|
||||||
self.raise_geo_restricted(countries=['AU'])
|
self.raise_geo_restricted(countries=['AU'])
|
||||||
formats = self._extract_m3u8_formats(m3u8_url, brightcove_id, 'mp4')
|
formats = self._extract_m3u8_formats(m3u8_url, content_id, 'mp4')
|
||||||
self._sort_formats(formats)
|
self._sort_formats(formats)
|
||||||
|
|
||||||
return {
|
return {
|
||||||
# '_type': 'url_transparent',
|
|
||||||
# 'url': brightcove_url,
|
|
||||||
'formats': formats,
|
'formats': formats,
|
||||||
'id': brightcove_id,
|
'id': data.get('altId') or content_id,
|
||||||
'title': video.get('title') or metadata.get('pageContentName') or metadata['showContentName'],
|
'title': data.get('title'),
|
||||||
'description': video.get('description'),
|
'description': data.get('description'),
|
||||||
'age_limit': parse_age_limit(video.get('showRatingClassification') or metadata.get('showProgramClassification')),
|
'age_limit': self._AUS_AGES[data.get('classification')],
|
||||||
'series': metadata.get('showName'),
|
'series': data.get('showName'),
|
||||||
'season': metadata.get('showContentSeason'),
|
'season': data.get('showContentSeason'),
|
||||||
'timestamp': parse_iso8601(metadata.get('contentPublishDate') or metadata.get('pageContentPublishDate')),
|
'timestamp': data.get('published'),
|
||||||
'thumbnail': video.get('poster'),
|
'thumbnail': data.get('imageUrl'),
|
||||||
|
'uploader': 'Channel 10',
|
||||||
'uploader_id': '2199827728001',
|
'uploader_id': '2199827728001',
|
||||||
# 'ie_key': 'BrightcoveNew',
|
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -153,7 +153,7 @@ class ToggleIE(InfoExtractor):
|
|||||||
})
|
})
|
||||||
if not formats:
|
if not formats:
|
||||||
for meta in (info.get('Metas') or []):
|
for meta in (info.get('Metas') or []):
|
||||||
if (not self._downloader.params.get('allow_unplayable_formats')
|
if (not self.get_param('allow_unplayable_formats')
|
||||||
and meta.get('Key') == 'Encryption' and meta.get('Value') == '1'):
|
and meta.get('Key') == 'Encryption' and meta.get('Value') == '1'):
|
||||||
self.raise_no_formats(
|
self.raise_no_formats(
|
||||||
'This video is DRM protected.', expected=True)
|
'This video is DRM protected.', expected=True)
|
||||||
|
|||||||
@@ -74,7 +74,7 @@ class TouTvIE(RadioCanadaIE):
|
|||||||
})
|
})
|
||||||
# IsDrm does not necessarily mean the video is DRM protected (see
|
# IsDrm does not necessarily mean the video is DRM protected (see
|
||||||
# https://github.com/ytdl-org/youtube-dl/issues/13994).
|
# https://github.com/ytdl-org/youtube-dl/issues/13994).
|
||||||
if not self._downloader.params.get('allow_unplayable_formats') and metadata.get('IsDrm'):
|
if not self.get_param('allow_unplayable_formats') and metadata.get('IsDrm'):
|
||||||
self.report_warning('This video is probably DRM protected.', path)
|
self.report_warning('This video is probably DRM protected.', path)
|
||||||
video_id = metadata['IdMedia']
|
video_id = metadata['IdMedia']
|
||||||
details = metadata['Details']
|
details = metadata['Details']
|
||||||
|
|||||||
@@ -69,7 +69,7 @@ class TVNowBaseIE(InfoExtractor):
|
|||||||
if formats:
|
if formats:
|
||||||
break
|
break
|
||||||
else:
|
else:
|
||||||
if not self._downloader.params.get('allow_unplayable_formats') and info.get('isDrm'):
|
if not self.get_param('allow_unplayable_formats') and info.get('isDrm'):
|
||||||
raise ExtractorError(
|
raise ExtractorError(
|
||||||
'Video %s is DRM protected' % video_id, expected=True)
|
'Video %s is DRM protected' % video_id, expected=True)
|
||||||
if info.get('geoblocked'):
|
if info.get('geoblocked'):
|
||||||
|
|||||||
@@ -59,7 +59,7 @@ class TwitCastingIE(InfoExtractor):
|
|||||||
def _real_extract(self, url):
|
def _real_extract(self, url):
|
||||||
uploader_id, video_id = re.match(self._VALID_URL, url).groups()
|
uploader_id, video_id = re.match(self._VALID_URL, url).groups()
|
||||||
|
|
||||||
video_password = self._downloader.params.get('videopassword')
|
video_password = self.get_param('videopassword')
|
||||||
request_data = None
|
request_data = None
|
||||||
if video_password:
|
if video_password:
|
||||||
request_data = urlencode_postdata({
|
request_data = urlencode_postdata({
|
||||||
|
|||||||
@@ -324,7 +324,7 @@ class VikiIE(VikiBaseIE):
|
|||||||
# Despite CODECS metadata in m3u8 all video-only formats
|
# Despite CODECS metadata in m3u8 all video-only formats
|
||||||
# are actually video+audio
|
# are actually video+audio
|
||||||
for f in m3u8_formats:
|
for f in m3u8_formats:
|
||||||
if not self._downloader.params.get('allow_unplayable_formats') and '_drm/index_' in f['url']:
|
if not self.get_param('allow_unplayable_formats') and '_drm/index_' in f['url']:
|
||||||
continue
|
continue
|
||||||
if f.get('acodec') == 'none' and f.get('vcodec') != 'none':
|
if f.get('acodec') == 'none' and f.get('vcodec') != 'none':
|
||||||
f['acodec'] = None
|
f['acodec'] = None
|
||||||
|
|||||||
@@ -76,7 +76,7 @@ class VimeoBaseInfoExtractor(InfoExtractor):
|
|||||||
raise ExtractorError('Unable to log in')
|
raise ExtractorError('Unable to log in')
|
||||||
|
|
||||||
def _get_video_password(self):
|
def _get_video_password(self):
|
||||||
password = self._downloader.params.get('videopassword')
|
password = self.get_param('videopassword')
|
||||||
if password is None:
|
if password is None:
|
||||||
raise ExtractorError(
|
raise ExtractorError(
|
||||||
'This video is protected by a password, use the --video-password option',
|
'This video is protected by a password, use the --video-password option',
|
||||||
@@ -603,7 +603,7 @@ class VimeoIE(VimeoBaseInfoExtractor):
|
|||||||
album_id, headers={'Authorization': 'jwt ' + jwt},
|
album_id, headers={'Authorization': 'jwt ' + jwt},
|
||||||
query={'fields': 'description,name,privacy'})
|
query={'fields': 'description,name,privacy'})
|
||||||
if try_get(album, lambda x: x['privacy']['view']) == 'password':
|
if try_get(album, lambda x: x['privacy']['view']) == 'password':
|
||||||
password = self._downloader.params.get('videopassword')
|
password = self.get_param('videopassword')
|
||||||
if not password:
|
if not password:
|
||||||
raise ExtractorError(
|
raise ExtractorError(
|
||||||
'This album is protected by a password, use the --video-password option',
|
'This album is protected by a password, use the --video-password option',
|
||||||
@@ -1058,7 +1058,7 @@ class VimeoAlbumIE(VimeoBaseInfoExtractor):
|
|||||||
query={'fields': 'description,name,privacy'})
|
query={'fields': 'description,name,privacy'})
|
||||||
hashed_pass = None
|
hashed_pass = None
|
||||||
if try_get(album, lambda x: x['privacy']['view']) == 'password':
|
if try_get(album, lambda x: x['privacy']['view']) == 'password':
|
||||||
password = self._downloader.params.get('videopassword')
|
password = self.get_param('videopassword')
|
||||||
if not password:
|
if not password:
|
||||||
raise ExtractorError(
|
raise ExtractorError(
|
||||||
'This album is protected by a password, use the --video-password option',
|
'This album is protected by a password, use the --video-password option',
|
||||||
|
|||||||
@@ -287,7 +287,7 @@ class ViuOTTIE(InfoExtractor):
|
|||||||
raise ExtractorError('This video is not available in your region.', expected=True)
|
raise ExtractorError('This video is not available in your region.', expected=True)
|
||||||
|
|
||||||
series_id = video_data.get('series_id')
|
series_id = video_data.get('series_id')
|
||||||
if not self._downloader.params.get('noplaylist') and not idata.get('force_noplaylist'):
|
if not self.get_param('noplaylist') and not idata.get('force_noplaylist'):
|
||||||
self.to_screen('Downloading playlist %s - add --no-playlist to just download video' % series_id)
|
self.to_screen('Downloading playlist %s - add --no-playlist to just download video' % series_id)
|
||||||
series = product_data.get('series', {})
|
series = product_data.get('series', {})
|
||||||
product = series.get('product')
|
product = series.get('product')
|
||||||
@@ -308,7 +308,7 @@ class ViuOTTIE(InfoExtractor):
|
|||||||
|
|
||||||
return self.playlist_result(entries, series_id, series.get('name'), series.get('description'))
|
return self.playlist_result(entries, series_id, series.get('name'), series.get('description'))
|
||||||
|
|
||||||
if self._downloader.params.get('noplaylist'):
|
if self.get_param('noplaylist'):
|
||||||
self.to_screen('Downloading just video %s because of --no-playlist' % video_id)
|
self.to_screen('Downloading just video %s because of --no-playlist' % video_id)
|
||||||
|
|
||||||
duration_limit = False
|
duration_limit = False
|
||||||
|
|||||||
@@ -300,6 +300,13 @@ class VKIE(VKBaseIE):
|
|||||||
'only_matching': True,
|
'only_matching': True,
|
||||||
}]
|
}]
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _extract_sibnet_urls(webpage):
|
||||||
|
# https://help.sibnet.ru/?sibnet_video_embed
|
||||||
|
return [unescapeHTML(mobj.group('url')) for mobj in re.finditer(
|
||||||
|
r'<iframe\b[^>]+\bsrc=(["\'])(?P<url>(?:https?:)?//video\.sibnet\.ru/shell\.php\?.*?\bvideoid=\d+.*?)\1',
|
||||||
|
webpage)]
|
||||||
|
|
||||||
def _real_extract(self, url):
|
def _real_extract(self, url):
|
||||||
mobj = re.match(self._VALID_URL, url)
|
mobj = re.match(self._VALID_URL, url)
|
||||||
video_id = mobj.group('videoid')
|
video_id = mobj.group('videoid')
|
||||||
@@ -408,6 +415,10 @@ class VKIE(VKBaseIE):
|
|||||||
if odnoklassniki_url:
|
if odnoklassniki_url:
|
||||||
return self.url_result(odnoklassniki_url, OdnoklassnikiIE.ie_key())
|
return self.url_result(odnoklassniki_url, OdnoklassnikiIE.ie_key())
|
||||||
|
|
||||||
|
sibnet_urls = self._extract_sibnet_urls(info_page)
|
||||||
|
if sibnet_urls:
|
||||||
|
return self.url_result(sibnet_urls[0])
|
||||||
|
|
||||||
m_opts = re.search(r'(?s)var\s+opts\s*=\s*({.+?});', info_page)
|
m_opts = re.search(r'(?s)var\s+opts\s*=\s*({.+?});', info_page)
|
||||||
if m_opts:
|
if m_opts:
|
||||||
m_opts_url = re.search(r"url\s*:\s*'((?!/\b)[^']+)", m_opts.group(1))
|
m_opts_url = re.search(r"url\s*:\s*'((?!/\b)[^']+)", m_opts.group(1))
|
||||||
|
|||||||
@@ -136,7 +136,7 @@ class VLiveIE(VLiveBaseIE):
|
|||||||
'author{nickname},channel{channelCode,channelName},officialVideo{commentCount,exposeStatus,likeCount,playCount,playTime,status,title,type,vodId},playlist{playlistSeq,totalCount,name}')
|
'author{nickname},channel{channelCode,channelName},officialVideo{commentCount,exposeStatus,likeCount,playCount,playTime,status,title,type,vodId},playlist{playlistSeq,totalCount,name}')
|
||||||
|
|
||||||
playlist = post.get('playlist')
|
playlist = post.get('playlist')
|
||||||
if not playlist or self._downloader.params.get('noplaylist'):
|
if not playlist or self.get_param('noplaylist'):
|
||||||
if playlist:
|
if playlist:
|
||||||
self.to_screen(
|
self.to_screen(
|
||||||
'Downloading just video %s because of --no-playlist'
|
'Downloading just video %s because of --no-playlist'
|
||||||
|
|||||||
@@ -41,7 +41,7 @@ class WakanimIE(InfoExtractor):
|
|||||||
m3u8_url = urljoin(url, self._search_regex(
|
m3u8_url = urljoin(url, self._search_regex(
|
||||||
r'file\s*:\s*(["\'])(?P<url>(?:(?!\1).)+)\1', webpage, 'm3u8 url',
|
r'file\s*:\s*(["\'])(?P<url>(?:(?!\1).)+)\1', webpage, 'm3u8 url',
|
||||||
group='url'))
|
group='url'))
|
||||||
if not self._downloader.params.get('allow_unplayable_formats'):
|
if not self.get_param('allow_unplayable_formats'):
|
||||||
# https://docs.microsoft.com/en-us/azure/media-services/previous/media-services-content-protection-overview#streaming-urls
|
# https://docs.microsoft.com/en-us/azure/media-services/previous/media-services-content-protection-overview#streaming-urls
|
||||||
encryption = self._search_regex(
|
encryption = self._search_regex(
|
||||||
r'encryption%3D(c(?:enc|bc(?:s-aapl)?))',
|
r'encryption%3D(c(?:enc|bc(?:s-aapl)?))',
|
||||||
|
|||||||
@@ -160,7 +160,7 @@ class YoukuIE(InfoExtractor):
|
|||||||
'client_ts': time.time() / 1000,
|
'client_ts': time.time() / 1000,
|
||||||
}
|
}
|
||||||
|
|
||||||
video_password = self._downloader.params.get('videopassword')
|
video_password = self.get_param('videopassword')
|
||||||
if video_password:
|
if video_password:
|
||||||
basic_data_params['password'] = video_password
|
basic_data_params['password'] = video_password
|
||||||
|
|
||||||
|
|||||||
+356
-140
@@ -67,7 +67,7 @@ class YoutubeBaseInfoExtractor(InfoExtractor):
|
|||||||
_TFA_URL = 'https://accounts.google.com/_/signin/challenge?hl=en&TL={0}'
|
_TFA_URL = 'https://accounts.google.com/_/signin/challenge?hl=en&TL={0}'
|
||||||
|
|
||||||
_RESERVED_NAMES = (
|
_RESERVED_NAMES = (
|
||||||
r'channel|c|user|playlist|watch|w|v|embed|e|watch_popup|'
|
r'channel|c|user|browse|playlist|watch|w|v|embed|e|watch_popup|'
|
||||||
r'movies|results|shared|hashtag|trending|feed|feeds|oembed|'
|
r'movies|results|shared|hashtag|trending|feed|feeds|oembed|'
|
||||||
r'storefront|oops|index|account|reporthistory|t/terms|about|upload|signin|logout')
|
r'storefront|oops|index|account|reporthistory|t/terms|about|upload|signin|logout')
|
||||||
|
|
||||||
@@ -85,12 +85,25 @@ class YoutubeBaseInfoExtractor(InfoExtractor):
|
|||||||
|
|
||||||
If _LOGIN_REQUIRED is set and no authentication was provided, an error is raised.
|
If _LOGIN_REQUIRED is set and no authentication was provided, an error is raised.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
def warn(message):
|
||||||
|
self.report_warning(message)
|
||||||
|
|
||||||
|
# username+password login is broken
|
||||||
|
if self._LOGIN_REQUIRED and self.get_param('cookiefile') is None:
|
||||||
|
self.raise_login_required(
|
||||||
|
'Login details are needed to download this content', method='cookies')
|
||||||
username, password = self._get_login_info()
|
username, password = self._get_login_info()
|
||||||
|
if username:
|
||||||
|
warn('Logging in using username and password is broken. %s' % self._LOGIN_HINTS['cookies'])
|
||||||
|
return
|
||||||
|
# Everything below this is broken!
|
||||||
|
|
||||||
# No authentication to be performed
|
# No authentication to be performed
|
||||||
if username is None:
|
if username is None:
|
||||||
if self._LOGIN_REQUIRED and self._downloader.params.get('cookiefile') is None:
|
if self._LOGIN_REQUIRED and self.get_param('cookiefile') is None:
|
||||||
raise ExtractorError('No login info available, needed for using %s.' % self.IE_NAME, expected=True)
|
raise ExtractorError('No login info available, needed for using %s.' % self.IE_NAME, expected=True)
|
||||||
# if self._downloader.params.get('cookiefile'): # TODO remove 'and False' later - too many people using outdated cookies and open issues, remind them.
|
# if self.get_param('cookiefile'): # TODO remove 'and False' later - too many people using outdated cookies and open issues, remind them.
|
||||||
# self.to_screen('[Cookies] Reminder - Make sure to always use up to date cookies!')
|
# self.to_screen('[Cookies] Reminder - Make sure to always use up to date cookies!')
|
||||||
return True
|
return True
|
||||||
|
|
||||||
@@ -126,9 +139,6 @@ class YoutubeBaseInfoExtractor(InfoExtractor):
|
|||||||
'Google-Accounts-XSRF': 1,
|
'Google-Accounts-XSRF': 1,
|
||||||
})
|
})
|
||||||
|
|
||||||
def warn(message):
|
|
||||||
self.report_warning(message)
|
|
||||||
|
|
||||||
lookup_req = [
|
lookup_req = [
|
||||||
username,
|
username,
|
||||||
None, [], None, 'US', None, None, 2, False, True,
|
None, [], None, 'US', None, None, 2, False, True,
|
||||||
@@ -398,6 +408,10 @@ class YoutubeBaseInfoExtractor(InfoExtractor):
|
|||||||
headers['X-Origin'] = 'https://www.youtube.com'
|
headers['X-Origin'] = 'https://www.youtube.com'
|
||||||
return headers
|
return headers
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def is_music_url(url):
|
||||||
|
return re.match(r'https?://music\.youtube\.com/', url) is not None
|
||||||
|
|
||||||
def _extract_video(self, renderer):
|
def _extract_video(self, renderer):
|
||||||
video_id = renderer.get('videoId')
|
video_id = renderer.get('videoId')
|
||||||
title = try_get(
|
title = try_get(
|
||||||
@@ -521,7 +535,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
|||||||
)? # all until now is optional -> you can pass the naked ID
|
)? # all until now is optional -> you can pass the naked ID
|
||||||
(?P<id>[0-9A-Za-z_-]{11}) # here is it! the YouTube video ID
|
(?P<id>[0-9A-Za-z_-]{11}) # here is it! the YouTube video ID
|
||||||
(?(1).+)? # if we found the ID, everything can follow
|
(?(1).+)? # if we found the ID, everything can follow
|
||||||
$""" % {
|
(?:\#|$)""" % {
|
||||||
'invidious': '|'.join(_INVIDIOUS_SITES),
|
'invidious': '|'.join(_INVIDIOUS_SITES),
|
||||||
}
|
}
|
||||||
_PLAYER_INFO_RE = (
|
_PLAYER_INFO_RE = (
|
||||||
@@ -1303,6 +1317,18 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
|||||||
'params': {
|
'params': {
|
||||||
'skip_download': True,
|
'skip_download': True,
|
||||||
},
|
},
|
||||||
|
}, {
|
||||||
|
# Has multiple audio streams
|
||||||
|
'url': 'WaOKSUlf4TM',
|
||||||
|
'only_matching': True
|
||||||
|
}, {
|
||||||
|
# Requires Premium: has format 141 when requested using YTM url
|
||||||
|
'url': 'https://music.youtube.com/watch?v=XclachpHxis',
|
||||||
|
'only_matching': True
|
||||||
|
}, {
|
||||||
|
# multiple subtitles with same lang_code
|
||||||
|
'url': 'https://www.youtube.com/watch?v=wsQiKKfKxug',
|
||||||
|
'only_matching': True,
|
||||||
},
|
},
|
||||||
]
|
]
|
||||||
|
|
||||||
@@ -1444,7 +1470,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
|||||||
)
|
)
|
||||||
self._player_cache[player_id] = func
|
self._player_cache[player_id] = func
|
||||||
func = self._player_cache[player_id]
|
func = self._player_cache[player_id]
|
||||||
if self._downloader.params.get('youtube_print_sig_code'):
|
if self.get_param('youtube_print_sig_code'):
|
||||||
self._print_sig_code(func, s)
|
self._print_sig_code(func, s)
|
||||||
return func(s)
|
return func(s)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
@@ -1674,7 +1700,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
|||||||
if not continuation:
|
if not continuation:
|
||||||
break
|
break
|
||||||
headers = self._generate_api_headers(ytcfg, identity_token, account_syncid, visitor_data)
|
headers = self._generate_api_headers(ytcfg, identity_token, account_syncid, visitor_data)
|
||||||
retries = self._downloader.params.get('extractor_retries', 3)
|
retries = self.get_param('extractor_retries', 3)
|
||||||
count = -1
|
count = -1
|
||||||
last_error = None
|
last_error = None
|
||||||
|
|
||||||
@@ -1843,11 +1869,47 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
|||||||
def _real_extract(self, url):
|
def _real_extract(self, url):
|
||||||
url, smuggled_data = unsmuggle_url(url, {})
|
url, smuggled_data = unsmuggle_url(url, {})
|
||||||
video_id = self._match_id(url)
|
video_id = self._match_id(url)
|
||||||
|
|
||||||
|
is_music_url = smuggled_data.get('is_music_url') or self.is_music_url(url)
|
||||||
|
|
||||||
base_url = self.http_scheme() + '//www.youtube.com/'
|
base_url = self.http_scheme() + '//www.youtube.com/'
|
||||||
webpage_url = base_url + 'watch?v=' + video_id
|
webpage_url = base_url + 'watch?v=' + video_id
|
||||||
webpage = self._download_webpage(
|
webpage = self._download_webpage(
|
||||||
webpage_url + '&bpctr=9999999999&has_verified=1', video_id, fatal=False)
|
webpage_url + '&bpctr=9999999999&has_verified=1', video_id, fatal=False)
|
||||||
|
|
||||||
|
def get_text(x):
|
||||||
|
if not x:
|
||||||
|
return
|
||||||
|
text = x.get('simpleText')
|
||||||
|
if text and isinstance(text, compat_str):
|
||||||
|
return text
|
||||||
|
runs = x.get('runs')
|
||||||
|
if not isinstance(runs, list):
|
||||||
|
return
|
||||||
|
return ''.join([r['text'] for r in runs if isinstance(r.get('text'), compat_str)])
|
||||||
|
|
||||||
|
ytm_streaming_data = {}
|
||||||
|
if is_music_url:
|
||||||
|
# we are forcing to use parse_json because 141 only appeared in get_video_info.
|
||||||
|
# el, c, cver, cplayer field required for 141(aac 256kbps) codec
|
||||||
|
# maybe paramter of youtube music player?
|
||||||
|
ytm_player_response = self._parse_json(try_get(compat_parse_qs(
|
||||||
|
self._download_webpage(
|
||||||
|
base_url + 'get_video_info', video_id,
|
||||||
|
'Fetching youtube music info webpage',
|
||||||
|
'unable to download youtube music info webpage', query={
|
||||||
|
'video_id': video_id,
|
||||||
|
'eurl': 'https://youtube.googleapis.com/v/' + video_id,
|
||||||
|
'el': 'detailpage',
|
||||||
|
'c': 'WEB_REMIX',
|
||||||
|
'cver': '0.1',
|
||||||
|
'cplayer': 'UNIPLAYER',
|
||||||
|
'html5': '1',
|
||||||
|
}, fatal=False)),
|
||||||
|
lambda x: x['player_response'][0],
|
||||||
|
compat_str) or '{}', video_id)
|
||||||
|
ytm_streaming_data = ytm_player_response.get('streamingData') or {}
|
||||||
|
|
||||||
player_response = None
|
player_response = None
|
||||||
if webpage:
|
if webpage:
|
||||||
player_response = self._extract_yt_initial_variable(
|
player_response = self._extract_yt_initial_variable(
|
||||||
@@ -1868,6 +1930,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
|||||||
'unable to download video info webpage', query={
|
'unable to download video info webpage', query={
|
||||||
'video_id': video_id,
|
'video_id': video_id,
|
||||||
'eurl': 'https://youtube.googleapis.com/v/' + video_id,
|
'eurl': 'https://youtube.googleapis.com/v/' + video_id,
|
||||||
|
'html5': '1',
|
||||||
}, fatal=False)),
|
}, fatal=False)),
|
||||||
lambda x: x['player_response'][0],
|
lambda x: x['player_response'][0],
|
||||||
compat_str) or '{}', video_id)
|
compat_str) or '{}', video_id)
|
||||||
@@ -1882,17 +1945,6 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
|||||||
return self.url_result(
|
return self.url_result(
|
||||||
trailer_video_id, self.ie_key(), trailer_video_id)
|
trailer_video_id, self.ie_key(), trailer_video_id)
|
||||||
|
|
||||||
def get_text(x):
|
|
||||||
if not x:
|
|
||||||
return
|
|
||||||
text = x.get('simpleText')
|
|
||||||
if text and isinstance(text, compat_str):
|
|
||||||
return text
|
|
||||||
runs = x.get('runs')
|
|
||||||
if not isinstance(runs, list):
|
|
||||||
return
|
|
||||||
return ''.join([r['text'] for r in runs if isinstance(r.get('text'), compat_str)])
|
|
||||||
|
|
||||||
search_meta = (
|
search_meta = (
|
||||||
lambda x: self._html_search_meta(x, webpage, default=None)) \
|
lambda x: self._html_search_meta(x, webpage, default=None)) \
|
||||||
if webpage else lambda x: None
|
if webpage else lambda x: None
|
||||||
@@ -1908,7 +1960,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
|||||||
video_description = video_details.get('shortDescription')
|
video_description = video_details.get('shortDescription')
|
||||||
|
|
||||||
if not smuggled_data.get('force_singlefeed', False):
|
if not smuggled_data.get('force_singlefeed', False):
|
||||||
if not self._downloader.params.get('noplaylist'):
|
if not self.get_param('noplaylist'):
|
||||||
multifeed_metadata_list = try_get(
|
multifeed_metadata_list = try_get(
|
||||||
player_response,
|
player_response,
|
||||||
lambda x: x['multicamera']['playerLegacyMulticameraRenderer']['metadataList'],
|
lambda x: x['multicamera']['playerLegacyMulticameraRenderer']['metadataList'],
|
||||||
@@ -1951,20 +2003,33 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
|||||||
else:
|
else:
|
||||||
self.to_screen('Downloading just video %s because of --no-playlist' % video_id)
|
self.to_screen('Downloading just video %s because of --no-playlist' % video_id)
|
||||||
|
|
||||||
formats = []
|
formats, itags, stream_ids = [], [], []
|
||||||
itags = []
|
|
||||||
itag_qualities = {}
|
itag_qualities = {}
|
||||||
player_url = None
|
player_url = None
|
||||||
q = qualities(['tiny', 'small', 'medium', 'large', 'hd720', 'hd1080', 'hd1440', 'hd2160', 'hd2880', 'highres'])
|
q = qualities([
|
||||||
|
'tiny', 'audio_quality_low', 'audio_quality_medium', 'audio_quality_high', # Audio only formats
|
||||||
|
'small', 'medium', 'large', 'hd720', 'hd1080', 'hd1440', 'hd2160', 'hd2880', 'highres'
|
||||||
|
])
|
||||||
|
|
||||||
streaming_data = player_response.get('streamingData') or {}
|
streaming_data = player_response.get('streamingData') or {}
|
||||||
streaming_formats = streaming_data.get('formats') or []
|
streaming_formats = streaming_data.get('formats') or []
|
||||||
streaming_formats.extend(streaming_data.get('adaptiveFormats') or [])
|
streaming_formats.extend(streaming_data.get('adaptiveFormats') or [])
|
||||||
|
streaming_formats.extend(ytm_streaming_data.get('formats') or [])
|
||||||
|
streaming_formats.extend(ytm_streaming_data.get('adaptiveFormats') or [])
|
||||||
|
|
||||||
for fmt in streaming_formats:
|
for fmt in streaming_formats:
|
||||||
if fmt.get('targetDurationSec') or fmt.get('drmFamilies'):
|
if fmt.get('targetDurationSec') or fmt.get('drmFamilies'):
|
||||||
continue
|
continue
|
||||||
|
|
||||||
itag = str_or_none(fmt.get('itag'))
|
itag = str_or_none(fmt.get('itag'))
|
||||||
|
audio_track = fmt.get('audioTrack') or {}
|
||||||
|
stream_id = '%s.%s' % (itag or '', audio_track.get('id', ''))
|
||||||
|
if stream_id in stream_ids:
|
||||||
|
continue
|
||||||
|
|
||||||
quality = fmt.get('quality')
|
quality = fmt.get('quality')
|
||||||
|
if quality == 'tiny' or not quality:
|
||||||
|
quality = fmt.get('audioQuality', '').lower() or quality
|
||||||
if itag and quality:
|
if itag and quality:
|
||||||
itag_qualities[itag] = quality
|
itag_qualities[itag] = quality
|
||||||
# FORMAT_STREAM_TYPE_OTF(otf=1) requires downloading the init fragment
|
# FORMAT_STREAM_TYPE_OTF(otf=1) requires downloading the init fragment
|
||||||
@@ -1994,19 +2059,22 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
|||||||
|
|
||||||
if itag:
|
if itag:
|
||||||
itags.append(itag)
|
itags.append(itag)
|
||||||
|
stream_ids.append(stream_id)
|
||||||
|
|
||||||
tbr = float_or_none(
|
tbr = float_or_none(
|
||||||
fmt.get('averageBitrate') or fmt.get('bitrate'), 1000)
|
fmt.get('averageBitrate') or fmt.get('bitrate'), 1000)
|
||||||
dct = {
|
dct = {
|
||||||
'asr': int_or_none(fmt.get('audioSampleRate')),
|
'asr': int_or_none(fmt.get('audioSampleRate')),
|
||||||
'filesize': int_or_none(fmt.get('contentLength')),
|
'filesize': int_or_none(fmt.get('contentLength')),
|
||||||
'format_id': itag,
|
'format_id': itag,
|
||||||
'format_note': fmt.get('qualityLabel') or quality,
|
'format_note': audio_track.get('displayName') or fmt.get('qualityLabel') or quality,
|
||||||
'fps': int_or_none(fmt.get('fps')),
|
'fps': int_or_none(fmt.get('fps')),
|
||||||
'height': int_or_none(fmt.get('height')),
|
'height': int_or_none(fmt.get('height')),
|
||||||
'quality': q(quality),
|
'quality': q(quality),
|
||||||
'tbr': tbr,
|
'tbr': tbr,
|
||||||
'url': fmt_url,
|
'url': fmt_url,
|
||||||
'width': fmt.get('width'),
|
'width': fmt.get('width'),
|
||||||
|
'language': audio_track.get('id', '').split('.')[0],
|
||||||
}
|
}
|
||||||
mimetype = fmt.get('mimeType')
|
mimetype = fmt.get('mimeType')
|
||||||
if mimetype:
|
if mimetype:
|
||||||
@@ -2030,38 +2098,37 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
|||||||
dct['container'] = dct['ext'] + '_dash'
|
dct['container'] = dct['ext'] + '_dash'
|
||||||
formats.append(dct)
|
formats.append(dct)
|
||||||
|
|
||||||
hls_manifest_url = streaming_data.get('hlsManifestUrl')
|
for sd in (streaming_data, ytm_streaming_data):
|
||||||
if hls_manifest_url:
|
hls_manifest_url = sd.get('hlsManifestUrl')
|
||||||
for f in self._extract_m3u8_formats(
|
if hls_manifest_url:
|
||||||
hls_manifest_url, video_id, 'mp4', fatal=False):
|
for f in self._extract_m3u8_formats(
|
||||||
itag = self._search_regex(
|
hls_manifest_url, video_id, 'mp4', fatal=False):
|
||||||
r'/itag/(\d+)', f['url'], 'itag', default=None)
|
itag = self._search_regex(
|
||||||
if itag:
|
r'/itag/(\d+)', f['url'], 'itag', default=None)
|
||||||
f['format_id'] = itag
|
if itag:
|
||||||
|
f['format_id'] = itag
|
||||||
formats.append(f)
|
formats.append(f)
|
||||||
|
|
||||||
if self._downloader.params.get('youtube_include_dash_manifest', True):
|
if self.get_param('youtube_include_dash_manifest', True):
|
||||||
dash_manifest_url = streaming_data.get('dashManifestUrl')
|
for sd in (streaming_data, ytm_streaming_data):
|
||||||
if dash_manifest_url:
|
dash_manifest_url = sd.get('dashManifestUrl')
|
||||||
for f in self._extract_mpd_formats(
|
if dash_manifest_url:
|
||||||
dash_manifest_url, video_id, fatal=False):
|
for f in self._extract_mpd_formats(
|
||||||
itag = f['format_id']
|
dash_manifest_url, video_id, fatal=False):
|
||||||
if itag in itags:
|
itag = f['format_id']
|
||||||
continue
|
if itag in itags:
|
||||||
if itag in itag_qualities:
|
continue
|
||||||
# Not actually usefull since the sorting is already done with "quality,res,fps,codec"
|
if itag in itag_qualities:
|
||||||
# but kept to maintain feature parity (and code similarity) with youtube-dl
|
f['quality'] = q(itag_qualities[itag])
|
||||||
# Remove if this causes any issues with sorting in future
|
filesize = int_or_none(self._search_regex(
|
||||||
f['quality'] = q(itag_qualities[itag])
|
r'/clen/(\d+)', f.get('fragment_base_url')
|
||||||
filesize = int_or_none(self._search_regex(
|
or f['url'], 'file size', default=None))
|
||||||
r'/clen/(\d+)', f.get('fragment_base_url')
|
if filesize:
|
||||||
or f['url'], 'file size', default=None))
|
f['filesize'] = filesize
|
||||||
if filesize:
|
formats.append(f)
|
||||||
f['filesize'] = filesize
|
|
||||||
formats.append(f)
|
|
||||||
|
|
||||||
if not formats:
|
if not formats:
|
||||||
if not self._downloader.params.get('allow_unplayable_formats') and streaming_data.get('licenseInfos'):
|
if not self.get_param('allow_unplayable_formats') and streaming_data.get('licenseInfos'):
|
||||||
self.raise_no_formats(
|
self.raise_no_formats(
|
||||||
'This video is DRM protected.', expected=True)
|
'This video is DRM protected.', expected=True)
|
||||||
pemr = try_get(
|
pemr = try_get(
|
||||||
@@ -2116,16 +2183,24 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
|||||||
if 'maxresdefault' in thumbnail_url:
|
if 'maxresdefault' in thumbnail_url:
|
||||||
thumbnail_url = thumbnail_url.split('?')[0]
|
thumbnail_url = thumbnail_url.split('?')[0]
|
||||||
thumbnails.append({
|
thumbnails.append({
|
||||||
'height': int_or_none(thumbnail.get('height')),
|
|
||||||
'url': thumbnail_url,
|
'url': thumbnail_url,
|
||||||
|
'height': int_or_none(thumbnail.get('height')),
|
||||||
'width': int_or_none(thumbnail.get('width')),
|
'width': int_or_none(thumbnail.get('width')),
|
||||||
|
'preference': 1 if 'maxresdefault' in thumbnail_url else -1
|
||||||
})
|
})
|
||||||
if thumbnails:
|
thumbnail_url = search_meta(['og:image', 'twitter:image'])
|
||||||
break
|
if thumbnail_url:
|
||||||
else:
|
thumbnails.append({
|
||||||
thumbnail = search_meta(['og:image', 'twitter:image'])
|
'url': thumbnail_url,
|
||||||
if thumbnail:
|
'preference': 1 if 'maxresdefault' in thumbnail_url else -1
|
||||||
thumbnails = [{'url': thumbnail}]
|
})
|
||||||
|
# All videos have a maxresdefault thumbnail, but sometimes it does not appear in the webpage
|
||||||
|
# See: https://github.com/ytdl-org/youtube-dl/issues/29049
|
||||||
|
thumbnails.append({
|
||||||
|
'url': 'https://i.ytimg.com/vi/%s/maxresdefault.jpg' % video_id,
|
||||||
|
'preference': 1,
|
||||||
|
})
|
||||||
|
self._remove_duplicate_formats(thumbnails)
|
||||||
|
|
||||||
category = microformat.get('category') or search_meta('genre')
|
category = microformat.get('category') or search_meta('genre')
|
||||||
channel_id = video_details.get('channelId') \
|
channel_id = video_details.get('channelId') \
|
||||||
@@ -2175,8 +2250,8 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
|||||||
lambda x: x['captions']['playerCaptionsTracklistRenderer'], dict)
|
lambda x: x['captions']['playerCaptionsTracklistRenderer'], dict)
|
||||||
subtitles = {}
|
subtitles = {}
|
||||||
if pctr:
|
if pctr:
|
||||||
def process_language(container, base_url, lang_code, query):
|
def process_language(container, base_url, lang_code, sub_name, query):
|
||||||
lang_subs = []
|
lang_subs = container.setdefault(lang_code, [])
|
||||||
for fmt in self._SUBTITLE_FORMATS:
|
for fmt in self._SUBTITLE_FORMATS:
|
||||||
query.update({
|
query.update({
|
||||||
'fmt': fmt,
|
'fmt': fmt,
|
||||||
@@ -2184,19 +2259,23 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
|||||||
lang_subs.append({
|
lang_subs.append({
|
||||||
'ext': fmt,
|
'ext': fmt,
|
||||||
'url': update_url_query(base_url, query),
|
'url': update_url_query(base_url, query),
|
||||||
|
'name': sub_name,
|
||||||
})
|
})
|
||||||
container[lang_code] = lang_subs
|
|
||||||
|
|
||||||
for caption_track in (pctr.get('captionTracks') or []):
|
for caption_track in (pctr.get('captionTracks') or []):
|
||||||
base_url = caption_track.get('baseUrl')
|
base_url = caption_track.get('baseUrl')
|
||||||
if not base_url:
|
if not base_url:
|
||||||
continue
|
continue
|
||||||
if caption_track.get('kind') != 'asr':
|
if caption_track.get('kind') != 'asr':
|
||||||
lang_code = caption_track.get('languageCode')
|
lang_code = (
|
||||||
|
remove_start(caption_track.get('vssId') or '', '.').replace('.', '-')
|
||||||
|
or caption_track.get('languageCode'))
|
||||||
if not lang_code:
|
if not lang_code:
|
||||||
continue
|
continue
|
||||||
process_language(
|
process_language(
|
||||||
subtitles, base_url, lang_code, {})
|
subtitles, base_url, lang_code,
|
||||||
|
try_get(caption_track, lambda x: x.get('name').get('simpleText')),
|
||||||
|
{})
|
||||||
continue
|
continue
|
||||||
automatic_captions = {}
|
automatic_captions = {}
|
||||||
for translation_language in (pctr.get('translationLanguages') or []):
|
for translation_language in (pctr.get('translationLanguages') or []):
|
||||||
@@ -2205,6 +2284,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
|||||||
continue
|
continue
|
||||||
process_language(
|
process_language(
|
||||||
automatic_captions, base_url, translation_language_code,
|
automatic_captions, base_url, translation_language_code,
|
||||||
|
try_get(translation_language, lambda x: x['languageName']['simpleText']),
|
||||||
{'tlang': translation_language_code})
|
{'tlang': translation_language_code})
|
||||||
info['automatic_captions'] = automatic_captions
|
info['automatic_captions'] = automatic_captions
|
||||||
info['subtitles'] = subtitles
|
info['subtitles'] = subtitles
|
||||||
@@ -2415,8 +2495,8 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
|||||||
is_unlisted=None if is_private is None else is_unlisted)
|
is_unlisted=None if is_private is None else is_unlisted)
|
||||||
|
|
||||||
# get xsrf for annotations or comments
|
# get xsrf for annotations or comments
|
||||||
get_annotations = self._downloader.params.get('writeannotations', False)
|
get_annotations = self.get_param('writeannotations', False)
|
||||||
get_comments = self._downloader.params.get('getcomments', False)
|
get_comments = self.get_param('getcomments', False)
|
||||||
if get_annotations or get_comments:
|
if get_annotations or get_comments:
|
||||||
xsrf_token = None
|
xsrf_token = None
|
||||||
ytcfg = self._extract_ytcfg(video_id, webpage)
|
ytcfg = self._extract_ytcfg(video_id, webpage)
|
||||||
@@ -2464,7 +2544,7 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
invidio\.us
|
invidio\.us
|
||||||
)/
|
)/
|
||||||
(?:
|
(?:
|
||||||
(?:channel|c|user)/|
|
(?P<channel_type>channel|c|user|browse)/|
|
||||||
(?P<not_channel>
|
(?P<not_channel>
|
||||||
feed/|hashtag/|
|
feed/|hashtag/|
|
||||||
(?:playlist|watch)\?.*?\blist=
|
(?:playlist|watch)\?.*?\blist=
|
||||||
@@ -2476,7 +2556,7 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
IE_NAME = 'youtube:tab'
|
IE_NAME = 'youtube:tab'
|
||||||
|
|
||||||
_TESTS = [{
|
_TESTS = [{
|
||||||
# playlists, multipage
|
'note': 'playlists, multipage',
|
||||||
'url': 'https://www.youtube.com/c/ИгорьКлейнер/playlists?view=1&flow=grid',
|
'url': 'https://www.youtube.com/c/ИгорьКлейнер/playlists?view=1&flow=grid',
|
||||||
'playlist_mincount': 94,
|
'playlist_mincount': 94,
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
@@ -2487,7 +2567,7 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
'uploader_id': 'UCqj7Cz7revf5maW9g5pgNcg',
|
'uploader_id': 'UCqj7Cz7revf5maW9g5pgNcg',
|
||||||
},
|
},
|
||||||
}, {
|
}, {
|
||||||
# playlists, multipage, different order
|
'note': 'playlists, multipage, different order',
|
||||||
'url': 'https://www.youtube.com/user/igorkle1/playlists?view=1&sort=dd',
|
'url': 'https://www.youtube.com/user/igorkle1/playlists?view=1&sort=dd',
|
||||||
'playlist_mincount': 94,
|
'playlist_mincount': 94,
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
@@ -2498,16 +2578,18 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
'uploader': 'Игорь Клейнер',
|
'uploader': 'Игорь Клейнер',
|
||||||
},
|
},
|
||||||
}, {
|
}, {
|
||||||
# playlists, series
|
'note': 'playlists, series',
|
||||||
'url': 'https://www.youtube.com/c/3blue1brown/playlists?view=50&sort=dd&shelf_id=3',
|
'url': 'https://www.youtube.com/c/3blue1brown/playlists?view=50&sort=dd&shelf_id=3',
|
||||||
'playlist_mincount': 5,
|
'playlist_mincount': 5,
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': 'UCYO_jab_esuFRV4b17AJtAw',
|
'id': 'UCYO_jab_esuFRV4b17AJtAw',
|
||||||
'title': '3Blue1Brown - Playlists',
|
'title': '3Blue1Brown - Playlists',
|
||||||
'description': 'md5:e1384e8a133307dd10edee76e875d62f',
|
'description': 'md5:e1384e8a133307dd10edee76e875d62f',
|
||||||
|
'uploader_id': 'UCYO_jab_esuFRV4b17AJtAw',
|
||||||
|
'uploader': '3Blue1Brown',
|
||||||
},
|
},
|
||||||
}, {
|
}, {
|
||||||
# playlists, singlepage
|
'note': 'playlists, singlepage',
|
||||||
'url': 'https://www.youtube.com/user/ThirstForScience/playlists',
|
'url': 'https://www.youtube.com/user/ThirstForScience/playlists',
|
||||||
'playlist_mincount': 4,
|
'playlist_mincount': 4,
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
@@ -2521,7 +2603,7 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
'url': 'https://www.youtube.com/c/ChristophLaimer/playlists',
|
'url': 'https://www.youtube.com/c/ChristophLaimer/playlists',
|
||||||
'only_matching': True,
|
'only_matching': True,
|
||||||
}, {
|
}, {
|
||||||
# basic, single video playlist
|
'note': 'basic, single video playlist',
|
||||||
'url': 'https://www.youtube.com/playlist?list=PL4lCao7KL_QFVb7Iudeipvc2BCavECqzc',
|
'url': 'https://www.youtube.com/playlist?list=PL4lCao7KL_QFVb7Iudeipvc2BCavECqzc',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'uploader_id': 'UCmlqkdCBesrv2Lak1mF_MxA',
|
'uploader_id': 'UCmlqkdCBesrv2Lak1mF_MxA',
|
||||||
@@ -2531,7 +2613,7 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
},
|
},
|
||||||
'playlist_count': 1,
|
'playlist_count': 1,
|
||||||
}, {
|
}, {
|
||||||
# empty playlist
|
'note': 'empty playlist',
|
||||||
'url': 'https://www.youtube.com/playlist?list=PL4lCao7KL_QFodcLWhDpGCYnngnHtQ-Xf',
|
'url': 'https://www.youtube.com/playlist?list=PL4lCao7KL_QFodcLWhDpGCYnngnHtQ-Xf',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'uploader_id': 'UCmlqkdCBesrv2Lak1mF_MxA',
|
'uploader_id': 'UCmlqkdCBesrv2Lak1mF_MxA',
|
||||||
@@ -2541,7 +2623,7 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
},
|
},
|
||||||
'playlist_count': 0,
|
'playlist_count': 0,
|
||||||
}, {
|
}, {
|
||||||
# Home tab
|
'note': 'Home tab',
|
||||||
'url': 'https://www.youtube.com/channel/UCKfVa3S1e4PHvxWcwyMMg8w/featured',
|
'url': 'https://www.youtube.com/channel/UCKfVa3S1e4PHvxWcwyMMg8w/featured',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': 'UCKfVa3S1e4PHvxWcwyMMg8w',
|
'id': 'UCKfVa3S1e4PHvxWcwyMMg8w',
|
||||||
@@ -2552,7 +2634,7 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
},
|
},
|
||||||
'playlist_mincount': 2,
|
'playlist_mincount': 2,
|
||||||
}, {
|
}, {
|
||||||
# Videos tab
|
'note': 'Videos tab',
|
||||||
'url': 'https://www.youtube.com/channel/UCKfVa3S1e4PHvxWcwyMMg8w/videos',
|
'url': 'https://www.youtube.com/channel/UCKfVa3S1e4PHvxWcwyMMg8w/videos',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': 'UCKfVa3S1e4PHvxWcwyMMg8w',
|
'id': 'UCKfVa3S1e4PHvxWcwyMMg8w',
|
||||||
@@ -2563,7 +2645,7 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
},
|
},
|
||||||
'playlist_mincount': 975,
|
'playlist_mincount': 975,
|
||||||
}, {
|
}, {
|
||||||
# Videos tab, sorted by popular
|
'note': 'Videos tab, sorted by popular',
|
||||||
'url': 'https://www.youtube.com/channel/UCKfVa3S1e4PHvxWcwyMMg8w/videos?view=0&sort=p&flow=grid',
|
'url': 'https://www.youtube.com/channel/UCKfVa3S1e4PHvxWcwyMMg8w/videos?view=0&sort=p&flow=grid',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': 'UCKfVa3S1e4PHvxWcwyMMg8w',
|
'id': 'UCKfVa3S1e4PHvxWcwyMMg8w',
|
||||||
@@ -2574,7 +2656,7 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
},
|
},
|
||||||
'playlist_mincount': 199,
|
'playlist_mincount': 199,
|
||||||
}, {
|
}, {
|
||||||
# Playlists tab
|
'note': 'Playlists tab',
|
||||||
'url': 'https://www.youtube.com/channel/UCKfVa3S1e4PHvxWcwyMMg8w/playlists',
|
'url': 'https://www.youtube.com/channel/UCKfVa3S1e4PHvxWcwyMMg8w/playlists',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': 'UCKfVa3S1e4PHvxWcwyMMg8w',
|
'id': 'UCKfVa3S1e4PHvxWcwyMMg8w',
|
||||||
@@ -2585,7 +2667,7 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
},
|
},
|
||||||
'playlist_mincount': 17,
|
'playlist_mincount': 17,
|
||||||
}, {
|
}, {
|
||||||
# Community tab
|
'note': 'Community tab',
|
||||||
'url': 'https://www.youtube.com/channel/UCKfVa3S1e4PHvxWcwyMMg8w/community',
|
'url': 'https://www.youtube.com/channel/UCKfVa3S1e4PHvxWcwyMMg8w/community',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': 'UCKfVa3S1e4PHvxWcwyMMg8w',
|
'id': 'UCKfVa3S1e4PHvxWcwyMMg8w',
|
||||||
@@ -2596,7 +2678,7 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
},
|
},
|
||||||
'playlist_mincount': 18,
|
'playlist_mincount': 18,
|
||||||
}, {
|
}, {
|
||||||
# Channels tab
|
'note': 'Channels tab',
|
||||||
'url': 'https://www.youtube.com/channel/UCKfVa3S1e4PHvxWcwyMMg8w/channels',
|
'url': 'https://www.youtube.com/channel/UCKfVa3S1e4PHvxWcwyMMg8w/channels',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': 'UCKfVa3S1e4PHvxWcwyMMg8w',
|
'id': 'UCKfVa3S1e4PHvxWcwyMMg8w',
|
||||||
@@ -2606,6 +2688,17 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
'uploader_id': 'UCKfVa3S1e4PHvxWcwyMMg8w',
|
'uploader_id': 'UCKfVa3S1e4PHvxWcwyMMg8w',
|
||||||
},
|
},
|
||||||
'playlist_mincount': 12,
|
'playlist_mincount': 12,
|
||||||
|
}, {
|
||||||
|
'note': 'Search tab',
|
||||||
|
'url': 'https://www.youtube.com/c/3blue1brown/search?query=linear%20algebra',
|
||||||
|
'playlist_mincount': 40,
|
||||||
|
'info_dict': {
|
||||||
|
'id': 'UCYO_jab_esuFRV4b17AJtAw',
|
||||||
|
'title': '3Blue1Brown - Search - linear algebra',
|
||||||
|
'description': 'md5:e1384e8a133307dd10edee76e875d62f',
|
||||||
|
'uploader': '3Blue1Brown',
|
||||||
|
'uploader_id': 'UCYO_jab_esuFRV4b17AJtAw',
|
||||||
|
},
|
||||||
}, {
|
}, {
|
||||||
'url': 'https://invidio.us/channel/UCmlqkdCBesrv2Lak1mF_MxA',
|
'url': 'https://invidio.us/channel/UCmlqkdCBesrv2Lak1mF_MxA',
|
||||||
'only_matching': True,
|
'only_matching': True,
|
||||||
@@ -2637,7 +2730,7 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
},
|
},
|
||||||
'playlist_mincount': 1123,
|
'playlist_mincount': 1123,
|
||||||
}, {
|
}, {
|
||||||
# even larger playlist, 8832 videos
|
'note': 'even larger playlist, 8832 videos',
|
||||||
'url': 'http://www.youtube.com/user/NASAgovVideo/videos',
|
'url': 'http://www.youtube.com/user/NASAgovVideo/videos',
|
||||||
'only_matching': True,
|
'only_matching': True,
|
||||||
}, {
|
}, {
|
||||||
@@ -2659,12 +2752,9 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
'uploader': 'Phim Siêu Nhân Nhật Bản',
|
'uploader': 'Phim Siêu Nhân Nhật Bản',
|
||||||
'uploader_id': 'UCTYLiWFZy8xtPwxFwX9rV7Q',
|
'uploader_id': 'UCTYLiWFZy8xtPwxFwX9rV7Q',
|
||||||
},
|
},
|
||||||
'playlist_mincount': 1400,
|
'playlist_mincount': 200,
|
||||||
'expected_warnings': [
|
|
||||||
'YouTube said: INFO - Unavailable videos are hidden',
|
|
||||||
]
|
|
||||||
}, {
|
}, {
|
||||||
'note': 'Playlist with unavailable videos in a later page',
|
'note': 'Playlist with unavailable videos in page 7',
|
||||||
'url': 'https://www.youtube.com/playlist?list=UU8l9frL61Yl5KFOl87nIm2w',
|
'url': 'https://www.youtube.com/playlist?list=UU8l9frL61Yl5KFOl87nIm2w',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'title': 'Uploads from BlankTV',
|
'title': 'Uploads from BlankTV',
|
||||||
@@ -2672,9 +2762,9 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
'uploader': 'BlankTV',
|
'uploader': 'BlankTV',
|
||||||
'uploader_id': 'UC8l9frL61Yl5KFOl87nIm2w',
|
'uploader_id': 'UC8l9frL61Yl5KFOl87nIm2w',
|
||||||
},
|
},
|
||||||
'playlist_mincount': 20000,
|
'playlist_mincount': 1000,
|
||||||
}, {
|
}, {
|
||||||
# https://github.com/ytdl-org/youtube-dl/issues/21844
|
'note': 'https://github.com/ytdl-org/youtube-dl/issues/21844',
|
||||||
'url': 'https://www.youtube.com/playlist?list=PLzH6n4zXuckpfMu_4Ff8E7Z1behQks5ba',
|
'url': 'https://www.youtube.com/playlist?list=PLzH6n4zXuckpfMu_4Ff8E7Z1behQks5ba',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'title': 'Data Analysis with Dr Mike Pound',
|
'title': 'Data Analysis with Dr Mike Pound',
|
||||||
@@ -2688,7 +2778,7 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
'url': 'https://invidio.us/playlist?list=PL4lCao7KL_QFVb7Iudeipvc2BCavECqzc',
|
'url': 'https://invidio.us/playlist?list=PL4lCao7KL_QFVb7Iudeipvc2BCavECqzc',
|
||||||
'only_matching': True,
|
'only_matching': True,
|
||||||
}, {
|
}, {
|
||||||
# Playlist URL that does not actually serve a playlist
|
'note': 'Playlist URL that does not actually serve a playlist',
|
||||||
'url': 'https://www.youtube.com/watch?v=FqZTN594JQw&list=PLMYEtVRpaqY00V9W81Cwmzp6N6vZqfUKD4',
|
'url': 'https://www.youtube.com/watch?v=FqZTN594JQw&list=PLMYEtVRpaqY00V9W81Cwmzp6N6vZqfUKD4',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': 'FqZTN594JQw',
|
'id': 'FqZTN594JQw',
|
||||||
@@ -2720,14 +2810,14 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
}, {
|
}, {
|
||||||
'url': 'https://www.youtube.com/channel/UCoMdktPbSTixAyNGwb-UYkQ/live',
|
'url': 'https://www.youtube.com/channel/UCoMdktPbSTixAyNGwb-UYkQ/live',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': '9Auq9mYxFEE',
|
'id': 'X1whbWASnNQ', # This will keep changing
|
||||||
'ext': 'mp4',
|
'ext': 'mp4',
|
||||||
'title': compat_str,
|
'title': compat_str,
|
||||||
'uploader': 'Sky News',
|
'uploader': 'Sky News',
|
||||||
'uploader_id': 'skynews',
|
'uploader_id': 'skynews',
|
||||||
'uploader_url': r're:https?://(?:www\.)?youtube\.com/user/skynews',
|
'uploader_url': r're:https?://(?:www\.)?youtube\.com/user/skynews',
|
||||||
'upload_date': '20191102',
|
'upload_date': r're:\d{8}',
|
||||||
'description': 'md5:85ddd75d888674631aaf9599a9a0b0ae',
|
'description': compat_str,
|
||||||
'categories': ['News & Politics'],
|
'categories': ['News & Politics'],
|
||||||
'tags': list,
|
'tags': list,
|
||||||
'like_count': int,
|
'like_count': int,
|
||||||
@@ -2736,6 +2826,7 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
'params': {
|
'params': {
|
||||||
'skip_download': True,
|
'skip_download': True,
|
||||||
},
|
},
|
||||||
|
'expected_warnings': ['Downloading just video ', 'Ignoring subtitle tracks found in '],
|
||||||
}, {
|
}, {
|
||||||
'url': 'https://www.youtube.com/user/TheYoungTurks/live',
|
'url': 'https://www.youtube.com/user/TheYoungTurks/live',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
@@ -2767,27 +2858,23 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
'url': 'https://www.youtube.com/feed/trending',
|
'url': 'https://www.youtube.com/feed/trending',
|
||||||
'only_matching': True,
|
'only_matching': True,
|
||||||
}, {
|
}, {
|
||||||
# needs auth
|
|
||||||
'url': 'https://www.youtube.com/feed/library',
|
'url': 'https://www.youtube.com/feed/library',
|
||||||
'only_matching': True,
|
'only_matching': True,
|
||||||
}, {
|
}, {
|
||||||
# needs auth
|
|
||||||
'url': 'https://www.youtube.com/feed/history',
|
'url': 'https://www.youtube.com/feed/history',
|
||||||
'only_matching': True,
|
'only_matching': True,
|
||||||
}, {
|
}, {
|
||||||
# needs auth
|
|
||||||
'url': 'https://www.youtube.com/feed/subscriptions',
|
'url': 'https://www.youtube.com/feed/subscriptions',
|
||||||
'only_matching': True,
|
'only_matching': True,
|
||||||
}, {
|
}, {
|
||||||
# needs auth
|
|
||||||
'url': 'https://www.youtube.com/feed/watch_later',
|
'url': 'https://www.youtube.com/feed/watch_later',
|
||||||
'only_matching': True,
|
'only_matching': True,
|
||||||
}, {
|
}, {
|
||||||
# no longer available?
|
'note': 'Recommended - redirects to home page',
|
||||||
'url': 'https://www.youtube.com/feed/recommended',
|
'url': 'https://www.youtube.com/feed/recommended',
|
||||||
'only_matching': True,
|
'only_matching': True,
|
||||||
}, {
|
}, {
|
||||||
# inline playlist with not always working continuations
|
'note': 'inline playlist with not always working continuations',
|
||||||
'url': 'https://www.youtube.com/watch?v=UC6u0Tct-Fo&list=PL36D642111D65BE7C',
|
'url': 'https://www.youtube.com/watch?v=UC6u0Tct-Fo&list=PL36D642111D65BE7C',
|
||||||
'only_matching': True,
|
'only_matching': True,
|
||||||
}, {
|
}, {
|
||||||
@@ -2815,6 +2902,60 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
}, {
|
}, {
|
||||||
'url': 'https://www.youtube.com/watch?list=PLW4dVinRY435CBE_JD3t-0SRXKfnZHS1P&feature=youtu.be&v=M9cJMXmQ_ZU',
|
'url': 'https://www.youtube.com/watch?list=PLW4dVinRY435CBE_JD3t-0SRXKfnZHS1P&feature=youtu.be&v=M9cJMXmQ_ZU',
|
||||||
'only_matching': True,
|
'only_matching': True,
|
||||||
|
}, {
|
||||||
|
'note': 'Requires Premium: should request additional YTM-info webpage (and have format 141) for videos in playlist',
|
||||||
|
'url': 'https://music.youtube.com/playlist?list=PLRBp0Fe2GpgmgoscNFLxNyBVSFVdYmFkq',
|
||||||
|
'only_matching': True
|
||||||
|
}, {
|
||||||
|
'note': '/browse/ should redirect to /channel/',
|
||||||
|
'url': 'https://music.youtube.com/browse/UC1a8OFewdjuLq6KlF8M_8Ng',
|
||||||
|
'only_matching': True
|
||||||
|
}, {
|
||||||
|
'note': 'VLPL, should redirect to playlist?list=PL...',
|
||||||
|
'url': 'https://music.youtube.com/browse/VLPLRBp0Fe2GpgmgoscNFLxNyBVSFVdYmFkq',
|
||||||
|
'info_dict': {
|
||||||
|
'id': 'PLRBp0Fe2GpgmgoscNFLxNyBVSFVdYmFkq',
|
||||||
|
'uploader': 'NoCopyrightSounds',
|
||||||
|
'description': 'Providing you with copyright free / safe music for gaming, live streaming, studying and more!',
|
||||||
|
'uploader_id': 'UC_aEa8K-EOJ3D6gOs7HcyNg',
|
||||||
|
'title': 'NCS Releases',
|
||||||
|
},
|
||||||
|
'playlist_mincount': 166,
|
||||||
|
}, {
|
||||||
|
'note': 'Topic, should redirect to playlist?list=UU...',
|
||||||
|
'url': 'https://music.youtube.com/browse/UC9ALqqC4aIeG5iDs7i90Bfw',
|
||||||
|
'info_dict': {
|
||||||
|
'id': 'UU9ALqqC4aIeG5iDs7i90Bfw',
|
||||||
|
'uploader_id': 'UC9ALqqC4aIeG5iDs7i90Bfw',
|
||||||
|
'title': 'Uploads from Royalty Free Music - Topic',
|
||||||
|
'uploader': 'Royalty Free Music - Topic',
|
||||||
|
},
|
||||||
|
'expected_warnings': [
|
||||||
|
'A channel/user page was given',
|
||||||
|
'The URL does not have a videos tab',
|
||||||
|
],
|
||||||
|
'playlist_mincount': 101,
|
||||||
|
}, {
|
||||||
|
'note': 'Topic without a UU playlist',
|
||||||
|
'url': 'https://www.youtube.com/channel/UCtFRv9O2AHqOZjjynzrv-xg',
|
||||||
|
'info_dict': {
|
||||||
|
'id': 'UCtFRv9O2AHqOZjjynzrv-xg',
|
||||||
|
'title': 'UCtFRv9O2AHqOZjjynzrv-xg',
|
||||||
|
},
|
||||||
|
'expected_warnings': [
|
||||||
|
'A channel/user page was given',
|
||||||
|
'The URL does not have a videos tab',
|
||||||
|
'Falling back to channel URL',
|
||||||
|
],
|
||||||
|
'playlist_mincount': 9,
|
||||||
|
}, {
|
||||||
|
'note': 'Youtube music Album',
|
||||||
|
'url': 'https://music.youtube.com/browse/MPREb_gTAcphH99wE',
|
||||||
|
'info_dict': {
|
||||||
|
'id': 'OLAK5uy_l1m0thk3g31NmIIz_vMIbWtyv7eZixlH0',
|
||||||
|
'title': 'Album - Royalty Free Music Library V2 (50 Songs)',
|
||||||
|
},
|
||||||
|
'playlist_count': 50,
|
||||||
}]
|
}]
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
@@ -3176,8 +3317,9 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
@staticmethod
|
@staticmethod
|
||||||
def _extract_selected_tab(tabs):
|
def _extract_selected_tab(tabs):
|
||||||
for tab in tabs:
|
for tab in tabs:
|
||||||
if try_get(tab, lambda x: x['tabRenderer']['selected'], bool):
|
renderer = dict_get(tab, ('tabRenderer', 'expandableTabRenderer')) or {}
|
||||||
return tab['tabRenderer']
|
if renderer.get('selected') is True:
|
||||||
|
return renderer
|
||||||
else:
|
else:
|
||||||
raise ExtractorError('Unable to find selected tab')
|
raise ExtractorError('Unable to find selected tab')
|
||||||
|
|
||||||
@@ -3251,6 +3393,7 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
try_get(data, lambda x: x['header']['hashtagHeaderRenderer']['hashtag']['simpleText'])
|
try_get(data, lambda x: x['header']['hashtagHeaderRenderer']['hashtag']['simpleText'])
|
||||||
or playlist_id)
|
or playlist_id)
|
||||||
title += format_field(selected_tab, 'title', ' - %s')
|
title += format_field(selected_tab, 'title', ' - %s')
|
||||||
|
title += format_field(selected_tab, 'expandedText', ' - %s')
|
||||||
|
|
||||||
metadata = {
|
metadata = {
|
||||||
'playlist_id': playlist_id,
|
'playlist_id': playlist_id,
|
||||||
@@ -3333,27 +3476,27 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
self._extract_mix_playlist(playlist, playlist_id, data, webpage),
|
self._extract_mix_playlist(playlist, playlist_id, data, webpage),
|
||||||
playlist_id=playlist_id, playlist_title=title)
|
playlist_id=playlist_id, playlist_title=title)
|
||||||
|
|
||||||
def _extract_alerts(self, data, expected=False):
|
@staticmethod
|
||||||
|
def _extract_alerts(data):
|
||||||
def _real_extract_alerts():
|
for alert_dict in try_get(data, lambda x: x['alerts'], list) or []:
|
||||||
for alert_dict in try_get(data, lambda x: x['alerts'], list) or []:
|
if not isinstance(alert_dict, dict):
|
||||||
if not isinstance(alert_dict, dict):
|
continue
|
||||||
|
for alert in alert_dict.values():
|
||||||
|
alert_type = alert.get('type')
|
||||||
|
if not alert_type:
|
||||||
continue
|
continue
|
||||||
for alert in alert_dict.values():
|
message = try_get(alert, lambda x: x['text']['simpleText'], compat_str) or ''
|
||||||
alert_type = alert.get('type')
|
if message:
|
||||||
if not alert_type:
|
yield alert_type, message
|
||||||
continue
|
for run in try_get(alert, lambda x: x['text']['runs'], list) or []:
|
||||||
message = try_get(alert, lambda x: x['text']['simpleText'], compat_str) or ''
|
message += try_get(run, lambda x: x['text'], compat_str)
|
||||||
if message:
|
if message:
|
||||||
yield alert_type, message
|
yield alert_type, message
|
||||||
for run in try_get(alert, lambda x: x['text']['runs'], list) or []:
|
|
||||||
message += try_get(run, lambda x: x['text'], compat_str)
|
|
||||||
if message:
|
|
||||||
yield alert_type, message
|
|
||||||
|
|
||||||
|
def _report_alerts(self, alerts, expected=True):
|
||||||
errors = []
|
errors = []
|
||||||
warnings = []
|
warnings = []
|
||||||
for alert_type, alert_message in _real_extract_alerts():
|
for alert_type, alert_message in alerts:
|
||||||
if alert_type.lower() == 'error':
|
if alert_type.lower() == 'error':
|
||||||
errors.append([alert_type, alert_message])
|
errors.append([alert_type, alert_message])
|
||||||
else:
|
else:
|
||||||
@@ -3364,6 +3507,9 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
if errors:
|
if errors:
|
||||||
raise ExtractorError('YouTube said: %s' % errors[-1][1], expected=expected)
|
raise ExtractorError('YouTube said: %s' % errors[-1][1], expected=expected)
|
||||||
|
|
||||||
|
def _extract_and_report_alerts(self, data, *args, **kwargs):
|
||||||
|
return self._report_alerts(self._extract_alerts(data), *args, **kwargs)
|
||||||
|
|
||||||
def _reload_with_unavailable_videos(self, item_id, data, webpage):
|
def _reload_with_unavailable_videos(self, item_id, data, webpage):
|
||||||
"""
|
"""
|
||||||
Get playlist with unavailable videos if the 'show unavailable videos' button exists.
|
Get playlist with unavailable videos if the 'show unavailable videos' button exists.
|
||||||
@@ -3413,7 +3559,7 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
response = None
|
response = None
|
||||||
last_error = None
|
last_error = None
|
||||||
count = -1
|
count = -1
|
||||||
retries = self._downloader.params.get('extractor_retries', 3)
|
retries = self.get_param('extractor_retries', 3)
|
||||||
if check_get_keys is None:
|
if check_get_keys is None:
|
||||||
check_get_keys = []
|
check_get_keys = []
|
||||||
while count < retries:
|
while count < retries:
|
||||||
@@ -3442,7 +3588,7 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
|
|
||||||
else:
|
else:
|
||||||
# Youtube may send alerts if there was an issue with the continuation page
|
# Youtube may send alerts if there was an issue with the continuation page
|
||||||
self._extract_alerts(response, expected=False)
|
self._extract_and_report_alerts(response, expected=False)
|
||||||
if not check_get_keys or dict_get(response, check_get_keys):
|
if not check_get_keys or dict_get(response, check_get_keys):
|
||||||
break
|
break
|
||||||
# Youtube sometimes sends incomplete data
|
# Youtube sometimes sends incomplete data
|
||||||
@@ -3457,7 +3603,7 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
return response
|
return response
|
||||||
|
|
||||||
def _extract_webpage(self, url, item_id):
|
def _extract_webpage(self, url, item_id):
|
||||||
retries = self._downloader.params.get('extractor_retries', 3)
|
retries = self.get_param('extractor_retries', 3)
|
||||||
count = -1
|
count = -1
|
||||||
last_error = 'Incomplete yt initial data recieved'
|
last_error = 'Incomplete yt initial data recieved'
|
||||||
while count < retries:
|
while count < retries:
|
||||||
@@ -3470,52 +3616,124 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
|
|||||||
url, item_id,
|
url, item_id,
|
||||||
'Downloading webpage%s' % (' (retry #%d)' % count if count else ''))
|
'Downloading webpage%s' % (' (retry #%d)' % count if count else ''))
|
||||||
data = self._extract_yt_initial_data(item_id, webpage)
|
data = self._extract_yt_initial_data(item_id, webpage)
|
||||||
self._extract_alerts(data, expected=True)
|
|
||||||
if data.get('contents') or data.get('currentVideoEndpoint'):
|
if data.get('contents') or data.get('currentVideoEndpoint'):
|
||||||
break
|
break
|
||||||
|
# Extract alerts here only when there is error
|
||||||
|
self._extract_and_report_alerts(data)
|
||||||
if count >= retries:
|
if count >= retries:
|
||||||
raise ExtractorError(last_error)
|
raise ExtractorError(last_error)
|
||||||
return webpage, data
|
return webpage, data
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _smuggle_data(entries, data):
|
||||||
|
for entry in entries:
|
||||||
|
if data:
|
||||||
|
entry['url'] = smuggle_url(entry['url'], data)
|
||||||
|
yield entry
|
||||||
|
|
||||||
def _real_extract(self, url):
|
def _real_extract(self, url):
|
||||||
|
url, smuggled_data = unsmuggle_url(url, {})
|
||||||
|
if self.is_music_url(url):
|
||||||
|
smuggled_data['is_music_url'] = True
|
||||||
|
info_dict = self.__real_extract(url, smuggled_data)
|
||||||
|
if info_dict.get('entries'):
|
||||||
|
info_dict['entries'] = self._smuggle_data(info_dict['entries'], smuggled_data)
|
||||||
|
return info_dict
|
||||||
|
|
||||||
|
_url_re = re.compile(r'(?P<pre>%s)(?(channel_type)(?P<tab>/\w+))?(?P<post>.*)$' % _VALID_URL)
|
||||||
|
|
||||||
|
def __real_extract(self, url, smuggled_data):
|
||||||
item_id = self._match_id(url)
|
item_id = self._match_id(url)
|
||||||
url = compat_urlparse.urlunparse(
|
url = compat_urlparse.urlunparse(
|
||||||
compat_urlparse.urlparse(url)._replace(netloc='www.youtube.com'))
|
compat_urlparse.urlparse(url)._replace(netloc='www.youtube.com'))
|
||||||
compat_opts = self._downloader.params.get('compat_opts', [])
|
compat_opts = self.get_param('compat_opts', [])
|
||||||
|
|
||||||
# This is not matched in a channel page with a tab selected
|
def get_mobj(url):
|
||||||
mobj = re.match(r'(?P<pre>%s)(?P<post>/?(?![^#?]).*$)' % self._VALID_URL, url)
|
mobj = self._url_re.match(url).groupdict()
|
||||||
mobj = mobj.groupdict() if mobj else {}
|
mobj.update((k, '') for k, v in mobj.items() if v is None)
|
||||||
if mobj and not mobj.get('not_channel') and 'no-youtube-channel-redirect' not in compat_opts:
|
return mobj
|
||||||
|
|
||||||
|
mobj = get_mobj(url)
|
||||||
|
# Youtube returns incomplete data if tabname is not lower case
|
||||||
|
pre, tab, post, is_channel = mobj['pre'], mobj['tab'].lower(), mobj['post'], not mobj['not_channel']
|
||||||
|
|
||||||
|
if is_channel:
|
||||||
|
if smuggled_data.get('is_music_url'):
|
||||||
|
if item_id[:2] == 'VL':
|
||||||
|
# Youtube music VL channels have an equivalent playlist
|
||||||
|
item_id = item_id[2:]
|
||||||
|
pre, tab, post, is_channel = 'https://www.youtube.com/playlist?list=%s' % item_id, '', '', False
|
||||||
|
elif item_id[:2] == 'MP':
|
||||||
|
# Youtube music albums (/channel/MP...) have a OLAK playlist that can be extracted from the webpage
|
||||||
|
item_id = self._search_regex(
|
||||||
|
r'\\x22audioPlaylistId\\x22:\\x22([0-9A-Za-z_-]+)\\x22',
|
||||||
|
self._download_webpage('https://music.youtube.com/channel/%s' % item_id, item_id),
|
||||||
|
'playlist id')
|
||||||
|
pre, tab, post, is_channel = 'https://www.youtube.com/playlist?list=%s' % item_id, '', '', False
|
||||||
|
elif mobj['channel_type'] == 'browse':
|
||||||
|
# Youtube music /browse/ should be changed to /channel/
|
||||||
|
pre = 'https://www.youtube.com/channel/%s' % item_id
|
||||||
|
if is_channel and not tab and 'no-youtube-channel-redirect' not in compat_opts:
|
||||||
|
# Home URLs should redirect to /videos/
|
||||||
self.report_warning(
|
self.report_warning(
|
||||||
'A channel/user page was given. All the channel\'s videos will be downloaded. '
|
'A channel/user page was given. All the channel\'s videos will be downloaded. '
|
||||||
'To download only the videos in the home page, add a "/featured" to the URL')
|
'To download only the videos in the home page, add a "/featured" to the URL')
|
||||||
url = '%s/videos%s' % (mobj.get('pre'), mobj.get('post') or '')
|
tab = '/videos'
|
||||||
|
|
||||||
|
url = ''.join((pre, tab, post))
|
||||||
|
mobj = get_mobj(url)
|
||||||
|
|
||||||
# Handle both video/playlist URLs
|
# Handle both video/playlist URLs
|
||||||
qs = parse_qs(url)
|
qs = parse_qs(url)
|
||||||
video_id = qs.get('v', [None])[0]
|
video_id = qs.get('v', [None])[0]
|
||||||
playlist_id = qs.get('list', [None])[0]
|
playlist_id = qs.get('list', [None])[0]
|
||||||
|
|
||||||
if not video_id and (mobj.get('not_channel') or '').startswith('watch'):
|
if not video_id and mobj['not_channel'].startswith('watch'):
|
||||||
if not playlist_id:
|
if not playlist_id:
|
||||||
# If there is neither video or playlist ids,
|
# If there is neither video or playlist ids, youtube redirects to home page, which is undesirable
|
||||||
# youtube redirects to home page, which is undesirable
|
|
||||||
raise ExtractorError('Unable to recognize tab page')
|
raise ExtractorError('Unable to recognize tab page')
|
||||||
|
# Common mistake: https://www.youtube.com/watch?list=playlist_id
|
||||||
self.report_warning('A video URL was given without video ID. Trying to download playlist %s' % playlist_id)
|
self.report_warning('A video URL was given without video ID. Trying to download playlist %s' % playlist_id)
|
||||||
url = 'https://www.youtube.com/playlist?list=%s' % playlist_id
|
url = 'https://www.youtube.com/playlist?list=%s' % playlist_id
|
||||||
|
mobj = get_mobj(url)
|
||||||
|
|
||||||
if video_id and playlist_id:
|
if video_id and playlist_id:
|
||||||
if self._downloader.params.get('noplaylist'):
|
if self.get_param('noplaylist'):
|
||||||
self.to_screen('Downloading just video %s because of --no-playlist' % video_id)
|
self.to_screen('Downloading just video %s because of --no-playlist' % video_id)
|
||||||
return self.url_result(video_id, ie=YoutubeIE.ie_key(), video_id=video_id)
|
return self.url_result(video_id, ie=YoutubeIE.ie_key(), video_id=video_id)
|
||||||
self.to_screen('Downloading playlist %s; add --no-playlist to just download video %s' % (playlist_id, video_id))
|
self.to_screen('Downloading playlist %s; add --no-playlist to just download video %s' % (playlist_id, video_id))
|
||||||
|
|
||||||
webpage, data = self._extract_webpage(url, item_id)
|
webpage, data = self._extract_webpage(url, item_id)
|
||||||
|
|
||||||
|
tabs = try_get(
|
||||||
|
data, lambda x: x['contents']['twoColumnBrowseResultsRenderer']['tabs'], list)
|
||||||
|
if tabs:
|
||||||
|
selected_tab = self._extract_selected_tab(tabs)
|
||||||
|
tab_name = selected_tab.get('title', '')
|
||||||
|
if (mobj['tab'] == '/videos' and tab_name.lower() != mobj['tab'][1:]
|
||||||
|
and 'no-youtube-channel-redirect' not in compat_opts):
|
||||||
|
if not mobj['not_channel'] and item_id[:2] == 'UC':
|
||||||
|
# Topic channels don't have /videos. Use the equivalent playlist instead
|
||||||
|
self.report_warning('The URL does not have a %s tab. Trying to redirect to playlist UU%s instead' % (mobj['tab'][1:], item_id[2:]))
|
||||||
|
pl_id = 'UU%s' % item_id[2:]
|
||||||
|
pl_url = 'https://www.youtube.com/playlist?list=%s%s' % (pl_id, mobj['post'])
|
||||||
|
try:
|
||||||
|
pl_webpage, pl_data = self._extract_webpage(pl_url, pl_id)
|
||||||
|
for alert_type, alert_message in self._extract_alerts(pl_data):
|
||||||
|
if alert_type == 'error':
|
||||||
|
raise ExtractorError('Youtube said: %s' % alert_message)
|
||||||
|
item_id, url, webpage, data = pl_id, pl_url, pl_webpage, pl_data
|
||||||
|
except ExtractorError:
|
||||||
|
self.report_warning('The playlist gave error. Falling back to channel URL')
|
||||||
|
else:
|
||||||
|
self.report_warning('The URL does not have a %s tab. %s is being downloaded instead' % (mobj['tab'][1:], tab_name))
|
||||||
|
|
||||||
|
self.write_debug('Final URL: %s' % url)
|
||||||
|
|
||||||
# YouTube sometimes provides a button to reload playlist with unavailable videos.
|
# YouTube sometimes provides a button to reload playlist with unavailable videos.
|
||||||
if 'no-youtube-unavailable-videos' not in compat_opts:
|
if 'no-youtube-unavailable-videos' not in compat_opts:
|
||||||
data = self._reload_with_unavailable_videos(item_id, data, webpage) or data
|
data = self._reload_with_unavailable_videos(item_id, data, webpage) or data
|
||||||
|
self._extract_and_report_alerts(data)
|
||||||
|
|
||||||
tabs = try_get(
|
tabs = try_get(
|
||||||
data, lambda x: x['contents']['twoColumnBrowseResultsRenderer']['tabs'], list)
|
data, lambda x: x['contents']['twoColumnBrowseResultsRenderer']['tabs'], list)
|
||||||
@@ -3612,12 +3830,13 @@ class YoutubePlaylistIE(InfoExtractor):
|
|||||||
|
|
||||||
def _real_extract(self, url):
|
def _real_extract(self, url):
|
||||||
playlist_id = self._match_id(url)
|
playlist_id = self._match_id(url)
|
||||||
qs = parse_qs(url)
|
is_music_url = self.is_music_url(url)
|
||||||
if not qs:
|
url = update_url_query(
|
||||||
qs = {'list': playlist_id}
|
'https://www.youtube.com/playlist',
|
||||||
return self.url_result(
|
parse_qs(url) or {'list': playlist_id})
|
||||||
update_url_query('https://www.youtube.com/playlist', qs),
|
if is_music_url:
|
||||||
ie=YoutubeTabIE.ie_key(), video_id=playlist_id)
|
url = smuggle_url(url, {'is_music_url': True})
|
||||||
|
return self.url_result(url, ie=YoutubeTabIE.ie_key(), video_id=playlist_id)
|
||||||
|
|
||||||
|
|
||||||
class YoutubeYtBeIE(InfoExtractor):
|
class YoutubeYtBeIE(InfoExtractor):
|
||||||
@@ -3811,9 +4030,6 @@ class YoutubeFeedsInfoExtractor(YoutubeTabIE):
|
|||||||
def IE_NAME(self):
|
def IE_NAME(self):
|
||||||
return 'youtube:%s' % self._FEED_NAME
|
return 'youtube:%s' % self._FEED_NAME
|
||||||
|
|
||||||
def _real_initialize(self):
|
|
||||||
self._login()
|
|
||||||
|
|
||||||
def _real_extract(self, url):
|
def _real_extract(self, url):
|
||||||
return self.url_result(
|
return self.url_result(
|
||||||
'https://www.youtube.com/feed/%s' % self._FEED_NAME,
|
'https://www.youtube.com/feed/%s' % self._FEED_NAME,
|
||||||
|
|||||||
@@ -35,7 +35,7 @@ class ZoomIE(InfoExtractor):
|
|||||||
except ExtractorError:
|
except ExtractorError:
|
||||||
form = None
|
form = None
|
||||||
if form:
|
if form:
|
||||||
password = self._downloader.params.get('videopassword')
|
password = self.get_param('videopassword')
|
||||||
if not password:
|
if not password:
|
||||||
raise ExtractorError(
|
raise ExtractorError(
|
||||||
'This video is protected by a passcode, use the --video-password option', expected=True)
|
'This video is protected by a passcode, use the --video-password option', expected=True)
|
||||||
|
|||||||
+38
-25
@@ -107,13 +107,15 @@ def parseOpts(overrideArguments=None):
|
|||||||
|
|
||||||
return ''.join(opts)
|
return ''.join(opts)
|
||||||
|
|
||||||
def _comma_separated_values_options_callback(option, opt_str, value, parser, prepend=True):
|
def _list_from_options_callback(option, opt_str, value, parser, append=True, delim=','):
|
||||||
|
# append can be True, False or -1 (prepend)
|
||||||
|
current = getattr(parser.values, option.dest) if append else []
|
||||||
|
value = [value] if delim is None else value.split(delim)
|
||||||
setattr(
|
setattr(
|
||||||
parser.values, option.dest,
|
parser.values, option.dest,
|
||||||
value.split(',') if not prepend
|
current + value if append is True else value + current)
|
||||||
else value.split(',') + getattr(parser.values, option.dest))
|
|
||||||
|
|
||||||
def _dict_from_multiple_values_options_callback(
|
def _dict_from_options_callback(
|
||||||
option, opt_str, value, parser,
|
option, opt_str, value, parser,
|
||||||
allowed_keys=r'[\w-]+', delimiter=':', default_key=None, process=None, multiple_keys=True):
|
allowed_keys=r'[\w-]+', delimiter=':', default_key=None, process=None, multiple_keys=True):
|
||||||
|
|
||||||
@@ -232,7 +234,7 @@ def parseOpts(overrideArguments=None):
|
|||||||
general.add_option(
|
general.add_option(
|
||||||
'--compat-options',
|
'--compat-options',
|
||||||
metavar='OPTS', dest='compat_opts', default=[],
|
metavar='OPTS', dest='compat_opts', default=[],
|
||||||
action='callback', callback=_comma_separated_values_options_callback, type='str',
|
action='callback', callback=_list_from_options_callback, type='str',
|
||||||
help=(
|
help=(
|
||||||
'Options that can help keep compatibility with youtube-dl and youtube-dlc '
|
'Options that can help keep compatibility with youtube-dl and youtube-dlc '
|
||||||
'configurations by reverting some of the changes made in yt-dlp. '
|
'configurations by reverting some of the changes made in yt-dlp. '
|
||||||
@@ -465,8 +467,8 @@ def parseOpts(overrideArguments=None):
|
|||||||
help='Video format code, see "FORMAT SELECTION" for more details')
|
help='Video format code, see "FORMAT SELECTION" for more details')
|
||||||
video_format.add_option(
|
video_format.add_option(
|
||||||
'-S', '--format-sort', metavar='SORTORDER',
|
'-S', '--format-sort', metavar='SORTORDER',
|
||||||
dest='format_sort', default=[],
|
dest='format_sort', default=[], type='str', action='callback',
|
||||||
action='callback', callback=_comma_separated_values_options_callback, type='str',
|
callback=_list_from_options_callback, callback_kwargs={'append': -1},
|
||||||
help='Sort the formats by the fields given, see "Sorting Formats" for more details')
|
help='Sort the formats by the fields given, see "Sorting Formats" for more details')
|
||||||
video_format.add_option(
|
video_format.add_option(
|
||||||
'--format-sort-force', '--S-force',
|
'--format-sort-force', '--S-force',
|
||||||
@@ -576,7 +578,7 @@ def parseOpts(overrideArguments=None):
|
|||||||
subtitles.add_option(
|
subtitles.add_option(
|
||||||
'--sub-langs', '--srt-langs',
|
'--sub-langs', '--srt-langs',
|
||||||
action='callback', dest='subtitleslangs', metavar='LANGS', type='str',
|
action='callback', dest='subtitleslangs', metavar='LANGS', type='str',
|
||||||
default=[], callback=_comma_separated_values_options_callback,
|
default=[], callback=_list_from_options_callback,
|
||||||
help=(
|
help=(
|
||||||
'Languages of the subtitles to download (can be regex) or "all" separated by commas. (Eg: --sub-langs en.*,ja) '
|
'Languages of the subtitles to download (can be regex) or "all" separated by commas. (Eg: --sub-langs en.*,ja) '
|
||||||
'You can prefix the language code with a "-" to exempt it from the requested languages. (Eg: --sub-langs all,-live_chat) '
|
'You can prefix the language code with a "-" to exempt it from the requested languages. (Eg: --sub-langs all,-live_chat) '
|
||||||
@@ -678,7 +680,7 @@ def parseOpts(overrideArguments=None):
|
|||||||
downloader.add_option(
|
downloader.add_option(
|
||||||
'--downloader', '--external-downloader',
|
'--downloader', '--external-downloader',
|
||||||
dest='external_downloader', metavar='[PROTO:]NAME', default={}, type='str',
|
dest='external_downloader', metavar='[PROTO:]NAME', default={}, type='str',
|
||||||
action='callback', callback=_dict_from_multiple_values_options_callback,
|
action='callback', callback=_dict_from_options_callback,
|
||||||
callback_kwargs={
|
callback_kwargs={
|
||||||
'allowed_keys': 'http|ftp|m3u8|dash|rtsp|rtmp|mms',
|
'allowed_keys': 'http|ftp|m3u8|dash|rtsp|rtmp|mms',
|
||||||
'default_key': 'default',
|
'default_key': 'default',
|
||||||
@@ -695,7 +697,7 @@ def parseOpts(overrideArguments=None):
|
|||||||
downloader.add_option(
|
downloader.add_option(
|
||||||
'--downloader-args', '--external-downloader-args',
|
'--downloader-args', '--external-downloader-args',
|
||||||
metavar='NAME:ARGS', dest='external_downloader_args', default={}, type='str',
|
metavar='NAME:ARGS', dest='external_downloader_args', default={}, type='str',
|
||||||
action='callback', callback=_dict_from_multiple_values_options_callback,
|
action='callback', callback=_dict_from_options_callback,
|
||||||
callback_kwargs={
|
callback_kwargs={
|
||||||
'allowed_keys': '|'.join(list_external_downloaders()),
|
'allowed_keys': '|'.join(list_external_downloaders()),
|
||||||
'default_key': 'default',
|
'default_key': 'default',
|
||||||
@@ -731,7 +733,7 @@ def parseOpts(overrideArguments=None):
|
|||||||
workarounds.add_option(
|
workarounds.add_option(
|
||||||
'--add-header',
|
'--add-header',
|
||||||
metavar='FIELD:VALUE', dest='headers', default={}, type='str',
|
metavar='FIELD:VALUE', dest='headers', default={}, type='str',
|
||||||
action='callback', callback=_dict_from_multiple_values_options_callback,
|
action='callback', callback=_dict_from_options_callback,
|
||||||
callback_kwargs={'multiple_keys': False},
|
callback_kwargs={'multiple_keys': False},
|
||||||
help='Specify a custom HTTP header and its value, separated by a colon ":". You can use this option multiple times',
|
help='Specify a custom HTTP header and its value, separated by a colon ":". You can use this option multiple times',
|
||||||
)
|
)
|
||||||
@@ -786,38 +788,45 @@ def parseOpts(overrideArguments=None):
|
|||||||
'--skip-download', '--no-download',
|
'--skip-download', '--no-download',
|
||||||
action='store_true', dest='skip_download', default=False,
|
action='store_true', dest='skip_download', default=False,
|
||||||
help='Do not download the video but write all related files (Alias: --no-download)')
|
help='Do not download the video but write all related files (Alias: --no-download)')
|
||||||
|
verbosity.add_option(
|
||||||
|
'-O', '--print', metavar='TEMPLATE',
|
||||||
|
action='callback', dest='forceprint', type='str', default=[],
|
||||||
|
callback=_list_from_options_callback, callback_kwargs={'delim': None},
|
||||||
|
help=(
|
||||||
|
'Simulate, quiet but print the given fields. Either a field name '
|
||||||
|
'or similar formatting as the output template can be used'))
|
||||||
verbosity.add_option(
|
verbosity.add_option(
|
||||||
'-g', '--get-url',
|
'-g', '--get-url',
|
||||||
action='store_true', dest='geturl', default=False,
|
action='store_true', dest='geturl', default=False,
|
||||||
help='Simulate, quiet but print URL')
|
help=optparse.SUPPRESS_HELP)
|
||||||
verbosity.add_option(
|
verbosity.add_option(
|
||||||
'-e', '--get-title',
|
'-e', '--get-title',
|
||||||
action='store_true', dest='gettitle', default=False,
|
action='store_true', dest='gettitle', default=False,
|
||||||
help='Simulate, quiet but print title')
|
help=optparse.SUPPRESS_HELP)
|
||||||
verbosity.add_option(
|
verbosity.add_option(
|
||||||
'--get-id',
|
'--get-id',
|
||||||
action='store_true', dest='getid', default=False,
|
action='store_true', dest='getid', default=False,
|
||||||
help='Simulate, quiet but print id')
|
help=optparse.SUPPRESS_HELP)
|
||||||
verbosity.add_option(
|
verbosity.add_option(
|
||||||
'--get-thumbnail',
|
'--get-thumbnail',
|
||||||
action='store_true', dest='getthumbnail', default=False,
|
action='store_true', dest='getthumbnail', default=False,
|
||||||
help='Simulate, quiet but print thumbnail URL')
|
help=optparse.SUPPRESS_HELP)
|
||||||
verbosity.add_option(
|
verbosity.add_option(
|
||||||
'--get-description',
|
'--get-description',
|
||||||
action='store_true', dest='getdescription', default=False,
|
action='store_true', dest='getdescription', default=False,
|
||||||
help='Simulate, quiet but print video description')
|
help=optparse.SUPPRESS_HELP)
|
||||||
verbosity.add_option(
|
verbosity.add_option(
|
||||||
'--get-duration',
|
'--get-duration',
|
||||||
action='store_true', dest='getduration', default=False,
|
action='store_true', dest='getduration', default=False,
|
||||||
help='Simulate, quiet but print video length')
|
help=optparse.SUPPRESS_HELP)
|
||||||
verbosity.add_option(
|
verbosity.add_option(
|
||||||
'--get-filename',
|
'--get-filename',
|
||||||
action='store_true', dest='getfilename', default=False,
|
action='store_true', dest='getfilename', default=False,
|
||||||
help='Simulate, quiet but print output filename')
|
help=optparse.SUPPRESS_HELP)
|
||||||
verbosity.add_option(
|
verbosity.add_option(
|
||||||
'--get-format',
|
'--get-format',
|
||||||
action='store_true', dest='getformat', default=False,
|
action='store_true', dest='getformat', default=False,
|
||||||
help='Simulate, quiet but print output format')
|
help=optparse.SUPPRESS_HELP)
|
||||||
verbosity.add_option(
|
verbosity.add_option(
|
||||||
'-j', '--dump-json',
|
'-j', '--dump-json',
|
||||||
action='store_true', dest='dumpjson', default=False,
|
action='store_true', dest='dumpjson', default=False,
|
||||||
@@ -893,7 +902,7 @@ def parseOpts(overrideArguments=None):
|
|||||||
filesystem.add_option(
|
filesystem.add_option(
|
||||||
'-P', '--paths',
|
'-P', '--paths',
|
||||||
metavar='TYPES:PATH', dest='paths', default={}, type='str',
|
metavar='TYPES:PATH', dest='paths', default={}, type='str',
|
||||||
action='callback', callback=_dict_from_multiple_values_options_callback,
|
action='callback', callback=_dict_from_options_callback,
|
||||||
callback_kwargs={'allowed_keys': 'home|temp|%s' % '|'.join(OUTTMPL_TYPES.keys())},
|
callback_kwargs={'allowed_keys': 'home|temp|%s' % '|'.join(OUTTMPL_TYPES.keys())},
|
||||||
help=(
|
help=(
|
||||||
'The paths where the files should be downloaded. '
|
'The paths where the files should be downloaded. '
|
||||||
@@ -906,7 +915,7 @@ def parseOpts(overrideArguments=None):
|
|||||||
filesystem.add_option(
|
filesystem.add_option(
|
||||||
'-o', '--output',
|
'-o', '--output',
|
||||||
metavar='[TYPES:]TEMPLATE', dest='outtmpl', default={}, type='str',
|
metavar='[TYPES:]TEMPLATE', dest='outtmpl', default={}, type='str',
|
||||||
action='callback', callback=_dict_from_multiple_values_options_callback,
|
action='callback', callback=_dict_from_options_callback,
|
||||||
callback_kwargs={
|
callback_kwargs={
|
||||||
'allowed_keys': '|'.join(OUTTMPL_TYPES.keys()),
|
'allowed_keys': '|'.join(OUTTMPL_TYPES.keys()),
|
||||||
'default_key': 'default'
|
'default_key': 'default'
|
||||||
@@ -1039,11 +1048,15 @@ def parseOpts(overrideArguments=None):
|
|||||||
action='store_false', dest='clean_infojson',
|
action='store_false', dest='clean_infojson',
|
||||||
help='Write all fields to the infojson')
|
help='Write all fields to the infojson')
|
||||||
filesystem.add_option(
|
filesystem.add_option(
|
||||||
'--get-comments',
|
'--write-comments', '--get-comments',
|
||||||
action='store_true', dest='getcomments', default=False,
|
action='store_true', dest='getcomments', default=False,
|
||||||
help=(
|
help=(
|
||||||
'Retrieve video comments to be placed in the .info.json file. '
|
'Retrieve video comments to be placed in the infojson. '
|
||||||
'The comments are fetched even without this option if the extraction is known to be quick'))
|
'The comments are fetched even without this option if the extraction is known to be quick (Alias: --get-comments)'))
|
||||||
|
filesystem.add_option(
|
||||||
|
'--no-write-comments', '--no-get-comments',
|
||||||
|
action='store_true', dest='getcomments', default=False,
|
||||||
|
help='Do not retrieve video comments unless the extraction is known to be quick (Alias: --no-get-comments)')
|
||||||
filesystem.add_option(
|
filesystem.add_option(
|
||||||
'--load-info-json', '--load-info',
|
'--load-info-json', '--load-info',
|
||||||
dest='load_info_filename', metavar='FILE',
|
dest='load_info_filename', metavar='FILE',
|
||||||
@@ -1132,7 +1145,7 @@ def parseOpts(overrideArguments=None):
|
|||||||
postproc.add_option(
|
postproc.add_option(
|
||||||
'--postprocessor-args', '--ppa',
|
'--postprocessor-args', '--ppa',
|
||||||
metavar='NAME:ARGS', dest='postprocessor_args', default={}, type='str',
|
metavar='NAME:ARGS', dest='postprocessor_args', default={}, type='str',
|
||||||
action='callback', callback=_dict_from_multiple_values_options_callback,
|
action='callback', callback=_dict_from_options_callback,
|
||||||
callback_kwargs={
|
callback_kwargs={
|
||||||
'allowed_keys': r'\w+(?:\+\w+)?', 'default_key': 'default-compat',
|
'allowed_keys': r'\w+(?:\+\w+)?', 'default_key': 'default-compat',
|
||||||
'process': compat_shlex_split,
|
'process': compat_shlex_split,
|
||||||
|
|||||||
@@ -54,10 +54,9 @@ class PostProcessor(object):
|
|||||||
if self._downloader:
|
if self._downloader:
|
||||||
return self._downloader.report_error(text, *args, **kwargs)
|
return self._downloader.report_error(text, *args, **kwargs)
|
||||||
|
|
||||||
def write_debug(self, text, prefix=True, *args, **kwargs):
|
def write_debug(self, text, *args, **kwargs):
|
||||||
tag = '[debug] ' if prefix else ''
|
if self._downloader:
|
||||||
if self.get_param('verbose', False) and self._downloader:
|
return self._downloader.write_debug(text, *args, **kwargs)
|
||||||
return self._downloader.to_screen('%s%s' % (tag, text), *args, **kwargs)
|
|
||||||
|
|
||||||
def get_param(self, name, default=None, *args, **kwargs):
|
def get_param(self, name, default=None, *args, **kwargs):
|
||||||
if self._downloader:
|
if self._downloader:
|
||||||
|
|||||||
@@ -1,14 +1,16 @@
|
|||||||
# coding: utf-8
|
# coding: utf-8
|
||||||
from __future__ import unicode_literals
|
from __future__ import unicode_literals
|
||||||
|
|
||||||
|
import base64
|
||||||
|
import imghdr
|
||||||
import os
|
import os
|
||||||
import subprocess
|
import subprocess
|
||||||
import struct
|
|
||||||
import re
|
import re
|
||||||
import base64
|
|
||||||
|
|
||||||
try:
|
try:
|
||||||
import mutagen
|
from mutagen.oggvorbis import OggVorbis
|
||||||
|
from mutagen.oggopus import OggOpus
|
||||||
|
from mutagen.flac import Picture, FLAC
|
||||||
has_mutagen = True
|
has_mutagen = True
|
||||||
except ImportError:
|
except ImportError:
|
||||||
has_mutagen = False
|
has_mutagen = False
|
||||||
@@ -39,6 +41,23 @@ class EmbedThumbnailPP(FFmpegPostProcessor):
|
|||||||
FFmpegPostProcessor.__init__(self, downloader)
|
FFmpegPostProcessor.__init__(self, downloader)
|
||||||
self._already_have_thumbnail = already_have_thumbnail
|
self._already_have_thumbnail = already_have_thumbnail
|
||||||
|
|
||||||
|
def _get_thumbnail_resolution(self, filename, thumbnail_dict):
|
||||||
|
def guess():
|
||||||
|
width, height = thumbnail_dict.get('width'), thumbnail_dict.get('height')
|
||||||
|
if width and height:
|
||||||
|
return width, height
|
||||||
|
|
||||||
|
try:
|
||||||
|
size_regex = r',\s*(?P<w>\d+)x(?P<h>\d+)\s*[,\[]'
|
||||||
|
size_result = self.run_ffmpeg(filename, filename, ['-hide_banner'])
|
||||||
|
mobj = re.search(size_regex, size_result)
|
||||||
|
if mobj is None:
|
||||||
|
return guess()
|
||||||
|
except PostProcessingError as err:
|
||||||
|
self.report_warning('unable to find the thumbnail resolution; %s' % error_to_compat_str(err))
|
||||||
|
return guess()
|
||||||
|
return int(mobj.group('w')), int(mobj.group('h'))
|
||||||
|
|
||||||
def run(self, info):
|
def run(self, info):
|
||||||
filename = info['filepath']
|
filename = info['filepath']
|
||||||
temp_filename = prepend_extension(filename, 'temp')
|
temp_filename = prepend_extension(filename, 'temp')
|
||||||
@@ -135,34 +154,32 @@ class EmbedThumbnailPP(FFmpegPostProcessor):
|
|||||||
self.report_warning('The file format doesn\'t support embedding a thumbnail')
|
self.report_warning('The file format doesn\'t support embedding a thumbnail')
|
||||||
success = False
|
success = False
|
||||||
|
|
||||||
elif info['ext'] in ['ogg', 'opus']:
|
elif info['ext'] in ['ogg', 'opus', 'flac']:
|
||||||
if not has_mutagen:
|
if not has_mutagen:
|
||||||
raise EmbedThumbnailPPError('module mutagen was not found. Please install using `python -m pip install mutagen`')
|
raise EmbedThumbnailPPError('module mutagen was not found. Please install using `python -m pip install mutagen`')
|
||||||
|
|
||||||
self.to_screen('Adding thumbnail to "%s"' % filename)
|
self.to_screen('Adding thumbnail to "%s"' % filename)
|
||||||
|
|
||||||
size_regex = r',\s*(?P<w>\d+)x(?P<h>\d+)\s*[,\[]'
|
|
||||||
size_result = self.run_ffmpeg(thumbnail_filename, thumbnail_filename, ['-hide_banner'])
|
|
||||||
mobj = re.search(size_regex, size_result)
|
|
||||||
width, height = int(mobj.group('w')), int(mobj.group('h'))
|
|
||||||
mimetype = ('image/%s' % ('png' if thumbnail_ext == 'png' else 'jpeg')).encode('ascii')
|
|
||||||
|
|
||||||
# https://xiph.org/flac/format.html#metadata_block_picture
|
|
||||||
data = bytearray()
|
|
||||||
data += struct.pack('>II', 3, len(mimetype))
|
|
||||||
data += mimetype
|
|
||||||
data += struct.pack('>IIIIII', 0, width, height, 8, 0, os.stat(thumbnail_filename).st_size) # 32 if png else 24
|
|
||||||
|
|
||||||
fin = open(thumbnail_filename, "rb")
|
|
||||||
data += fin.read()
|
|
||||||
fin.close()
|
|
||||||
|
|
||||||
temp_filename = filename
|
temp_filename = filename
|
||||||
f = mutagen.File(temp_filename)
|
f = {'opus': OggOpus, 'flac': FLAC, 'ogg': OggVorbis}[info['ext']](filename)
|
||||||
f.tags['METADATA_BLOCK_PICTURE'] = base64.b64encode(data).decode('ascii')
|
|
||||||
|
pic = Picture()
|
||||||
|
pic.mime = 'image/%s' % imghdr.what(thumbnail_filename)
|
||||||
|
with open(thumbnail_filename, 'rb') as thumbfile:
|
||||||
|
pic.data = thumbfile.read()
|
||||||
|
pic.type = 3 # front cover
|
||||||
|
res = self._get_thumbnail_resolution(thumbnail_filename, info['thumbnails'][-1])
|
||||||
|
if res is not None:
|
||||||
|
pic.width, pic.height = res
|
||||||
|
|
||||||
|
if info['ext'] == 'flac':
|
||||||
|
f.add_picture(pic)
|
||||||
|
else:
|
||||||
|
# https://wiki.xiph.org/VorbisComment#METADATA_BLOCK_PICTURE
|
||||||
|
f['METADATA_BLOCK_PICTURE'] = base64.b64encode(pic.write()).decode('ascii')
|
||||||
f.save()
|
f.save()
|
||||||
|
|
||||||
else:
|
else:
|
||||||
raise EmbedThumbnailPPError('Supported filetypes for thumbnail embedding are: mp3, mkv/mka, ogg/opus, m4a/mp4/mov')
|
raise EmbedThumbnailPPError('Supported filetypes for thumbnail embedding are: mp3, mkv/mka, ogg/opus/flac, m4a/mp4/mov')
|
||||||
|
|
||||||
if success and temp_filename != filename:
|
if success and temp_filename != filename:
|
||||||
os.remove(encodeFilename(filename))
|
os.remove(encodeFilename(filename))
|
||||||
|
|||||||
@@ -474,8 +474,7 @@ class FFmpegEmbedSubtitlePP(FFmpegPostProcessor):
|
|||||||
filename = information['filepath']
|
filename = information['filepath']
|
||||||
|
|
||||||
ext = information['ext']
|
ext = information['ext']
|
||||||
sub_langs = []
|
sub_langs, sub_names, sub_filenames = [], [], []
|
||||||
sub_filenames = []
|
|
||||||
webm_vtt_warn = False
|
webm_vtt_warn = False
|
||||||
mp4_ass_warn = False
|
mp4_ass_warn = False
|
||||||
|
|
||||||
@@ -485,6 +484,7 @@ class FFmpegEmbedSubtitlePP(FFmpegPostProcessor):
|
|||||||
self.report_warning('JSON subtitles cannot be embedded')
|
self.report_warning('JSON subtitles cannot be embedded')
|
||||||
elif ext != 'webm' or ext == 'webm' and sub_ext == 'vtt':
|
elif ext != 'webm' or ext == 'webm' and sub_ext == 'vtt':
|
||||||
sub_langs.append(lang)
|
sub_langs.append(lang)
|
||||||
|
sub_names.append(sub_info.get('name'))
|
||||||
sub_filenames.append(sub_info['filepath'])
|
sub_filenames.append(sub_info['filepath'])
|
||||||
else:
|
else:
|
||||||
if not webm_vtt_warn and ext == 'webm' and sub_ext != 'vtt':
|
if not webm_vtt_warn and ext == 'webm' and sub_ext != 'vtt':
|
||||||
@@ -510,10 +510,13 @@ class FFmpegEmbedSubtitlePP(FFmpegPostProcessor):
|
|||||||
]
|
]
|
||||||
if information['ext'] == 'mp4':
|
if information['ext'] == 'mp4':
|
||||||
opts += ['-c:s', 'mov_text']
|
opts += ['-c:s', 'mov_text']
|
||||||
for (i, lang) in enumerate(sub_langs):
|
for i, (lang, name) in enumerate(zip(sub_langs, sub_names)):
|
||||||
opts.extend(['-map', '%d:0' % (i + 1)])
|
opts.extend(['-map', '%d:0' % (i + 1)])
|
||||||
lang_code = ISO639Utils.short2long(lang) or lang
|
lang_code = ISO639Utils.short2long(lang) or lang
|
||||||
opts.extend(['-metadata:s:s:%d' % i, 'language=%s' % lang_code])
|
opts.extend(['-metadata:s:s:%d' % i, 'language=%s' % lang_code])
|
||||||
|
if name:
|
||||||
|
opts.extend(['-metadata:s:s:%d' % i, 'handler_name=%s' % name,
|
||||||
|
'-metadata:s:s:%d' % i, 'title=%s' % name])
|
||||||
|
|
||||||
temp_filename = prepend_extension(filename, 'temp')
|
temp_filename = prepend_extension(filename, 'temp')
|
||||||
self.to_screen('Embedding subtitles in "%s"' % filename)
|
self.to_screen('Embedding subtitles in "%s"' % filename)
|
||||||
@@ -605,7 +608,8 @@ class FFmpegMetadataPP(FFmpegPostProcessor):
|
|||||||
in_filenames.append(metadata_filename)
|
in_filenames.append(metadata_filename)
|
||||||
options.extend(['-map_metadata', '1'])
|
options.extend(['-map_metadata', '1'])
|
||||||
|
|
||||||
if '__infojson_filename' in info and info['ext'] in ('mkv', 'mka'):
|
if ('no-attach-info-json' not in self.get_param('compat_opts', [])
|
||||||
|
and '__infojson_filename' in info and info['ext'] in ('mkv', 'mka')):
|
||||||
old_stream, new_stream = self.get_stream_number(
|
old_stream, new_stream = self.get_stream_number(
|
||||||
filename, ('tags', 'mimetype'), 'application/json')
|
filename, ('tags', 'mimetype'), 'application/json')
|
||||||
if old_stream is not None:
|
if old_stream is not None:
|
||||||
|
|||||||
@@ -1748,6 +1748,7 @@ DATE_FORMATS = (
|
|||||||
'%Y-%m-%d %H:%M',
|
'%Y-%m-%d %H:%M',
|
||||||
'%Y-%m-%d %H:%M:%S',
|
'%Y-%m-%d %H:%M:%S',
|
||||||
'%Y-%m-%d %H:%M:%S.%f',
|
'%Y-%m-%d %H:%M:%S.%f',
|
||||||
|
'%Y-%m-%d %H:%M:%S:%f',
|
||||||
'%d.%m.%Y %H:%M',
|
'%d.%m.%Y %H:%M',
|
||||||
'%d.%m.%Y %H.%M',
|
'%d.%m.%Y %H.%M',
|
||||||
'%Y-%m-%dT%H:%M:%SZ',
|
'%Y-%m-%dT%H:%M:%SZ',
|
||||||
@@ -4311,6 +4312,7 @@ OUTTMPL_TYPES = {
|
|||||||
'description': 'description',
|
'description': 'description',
|
||||||
'annotation': 'annotations.xml',
|
'annotation': 'annotations.xml',
|
||||||
'infojson': 'info.json',
|
'infojson': 'info.json',
|
||||||
|
'pl_thumbnail': None,
|
||||||
'pl_description': 'description',
|
'pl_description': 'description',
|
||||||
'pl_infojson': 'info.json',
|
'pl_infojson': 'info.json',
|
||||||
}
|
}
|
||||||
|
|||||||
+1
-1
@@ -1,3 +1,3 @@
|
|||||||
from __future__ import unicode_literals
|
from __future__ import unicode_literals
|
||||||
|
|
||||||
__version__ = '2021.04.22'
|
__version__ = '2021.05.11'
|
||||||
|
|||||||
@@ -1,2 +1,3 @@
|
|||||||
# flake8: noqa
|
# flake8: noqa
|
||||||
|
|
||||||
from .sample import SamplePluginIE
|
from .sample import SamplePluginIE
|
||||||
|
|||||||
@@ -1,11 +1,13 @@
|
|||||||
|
# coding: utf-8
|
||||||
|
|
||||||
from __future__ import unicode_literals
|
from __future__ import unicode_literals
|
||||||
|
|
||||||
# Don't use relative imports
|
# ⚠ Don't use relative imports
|
||||||
from yt_dlp.extractor.common import InfoExtractor
|
from yt_dlp.extractor.common import InfoExtractor
|
||||||
|
|
||||||
|
|
||||||
# See https://github.com/ytdl-org/youtube-dl#adding-support-for-a-new-site
|
# ℹ️ Instructions on making extractors can be found at:
|
||||||
# for instuctions on making extractors
|
# 🔗 https://github.com/ytdl-org/youtube-dl#adding-support-for-a-new-site
|
||||||
|
|
||||||
class SamplePluginIE(InfoExtractor):
|
class SamplePluginIE(InfoExtractor):
|
||||||
_WORKING = False
|
_WORKING = False
|
||||||
|
|||||||
Reference in New Issue
Block a user