Compare commits

...
245 Commits
Author SHA1 Message Date
github-actions 41bd0dc4d7 [version] update
Created by: pukkandan

:ci skip all :ci run dl
2023-02-17 12:31:30 +00:00
pukkandan a0a7c01542 Release 2023.02.17 2023-02-17 17:52:25 +05:30
pukkandan 45b2ee6f4f Update to ytdl-commit-2dd6c6e
[YouTube] Avoid crash if uploader_id extraction fails
https://github.com/ytdl-org/youtube-dl/commit/2dd6c6edd8e0fc5e45865b8e6d865e35147de772

Except:
    * 295736c9cba714fb5de7d1c3dd31d86e50091cf8 [jsinterp] Improve parsing
    * 384f632e8a9b61e864a26678d85b2b39933b9bae [ITV] Overhaul ITV extractor
    * 33db85c571304bbd6863e3407ad8d08764c9e53b [feat]: Add support to external downloader aria2p
2023-02-17 17:52:23 +05:30
pukkandan a538772969 [cleanup] Misc
Closes #5897
2023-02-17 17:52:22 +05:30
HobbyistDevandGitHub 30031be974 [extractor/tempo] Add IVXPlayer extractor (#5837)
Authored by: HobbyistDev
2023-02-17 14:46:46 +05:30
HobbyistDevandGitHub 9acca71237 [extractor/boxcast] Add extractor (#5983)
Authored by: HobbyistDev
Closes #5769
2023-02-17 14:35:46 +05:30
Henrik HeimbuergerandGitHub d50ea3ce5a [extractor/nebula] Remove broken cookie support (#5979)
Authored by: hheimbuerger
Closes #4002
2023-02-17 14:02:55 +05:30
bashonly c61cf091a5 [extractor/youtube] uploader_id includes @ with handle
Authored by: bashonly
2023-02-17 02:14:45 -06:00
Chris CarusoandGitHub f737fb16d8 [ExtractAudio] Handle outtmpl without ext (#6005)
Authored by: carusocr
Closes #5968
2023-02-17 13:36:15 +05:30
Friedrich RehrenandGitHub 5e1a54f63e [extractor/SportDeutschland] Fix extractor (#6041)
Authored by: FriedrichRehren
Closes #3005
2023-02-17 13:14:26 +05:30
HobbyistDevandGitHub 31c279a2a2 [extractor/hypergryph] Add extractor (#6094)
Authored by: HobbyistDev, bashonly
Closes #6052
2023-02-17 09:33:04 +05:30
HobbyistDevandGitHub a4ad59ff2d [extractor/anchorfm] Add episode extractor (#6092)
Authored by: HobbyistDev, bashonly
Closes #6081
2023-02-17 09:29:04 +05:30
Alex IonescuandGitHub b25d6cb963 [utils] Fix race condition in make_dir (#6089)
Authored by: aionescu
2023-02-17 08:59:32 +05:30
HobbyistDevandGitHub 3616300155 [extractor/yappy] Add extractor (#6111)
Authored by: HobbyistDev
Closes #3522
2023-02-17 08:49:24 +05:30
qbnuandGitHub e4a8b1769e [extractor/vocaroo] Add extractor (#6117)
Authored by: qbnu, SuperSonicHub1
Closes #6152
2023-02-17 08:48:07 +05:30
JChris246andGitHub da880559a6 [extractor/ebay] Add extractor (#6170)
Closes #6134
Authored by: JChris246
2023-02-17 08:44:33 +05:30
Felix YanandGitHub 65e5c021e7 [utils] Don't use Content-length with encoding (#6176)
Authored by: felixonmars
Closes #3772, #6178
2023-02-17 08:38:45 +05:30
OIRNOIRandGitHub a9189510ba [extractor/nitter] Update instance list (#6236)
Authored by: OIRNOIR
2023-02-17 08:36:16 +05:30
HobbyistDevandGitHub 10fd9e6ee8 [extractor/odkmedia] Add OnDemandChinaEpisodeIE (#6116)
Authored by: HobbyistDev, pukkandan
2023-02-17 08:30:07 +05:30
HobbyistDevandGitHub 72671a212d [extractor/viu] Add ViuOTTIndonesiaIE extractor (#6099)
Authored by: HobbyistDev
Closes #1757
2023-02-17 08:27:52 +05:30
Siddhartha SahuandGitHub 376aa24b15 Improve default subtitle language selection (#6240)
Authored by: sdht0
2023-02-17 01:25:01 +05:30
Simon SawickiandGitHub c9d14bd22a [extractor/crunchyroll] Fix incorrect premium-only error
Closes #6234

Authored by: Grub4K
2023-02-16 15:54:11 +01:00
bashonly 149eb0bbf3 [extractor/youtube] Fix uploader_id extraction
Closes #6247
Authored by: bashonly
2023-02-16 08:51:45 -06:00
pukkandan 9ebac35577 Bugfix for 39f32f1715
when `--ignore-no-formats-error`
2023-02-16 17:06:54 +05:30
bashonlyandGitHub 8b37c58f8b [extractor/nfl] Add NFLPlus extractors (#6222)
Closes #6165
Authored by: bashonly
2023-02-14 02:57:24 +00:00
Greg SadetskyandGitHub d3bb187f01 [extractor/NZOnScreen] Add extractor (#6208)
Authored by: gregsadetsky, pukkandan
Closes #6193
2023-02-14 08:22:27 +05:30
pukkandan 44699d10dc [extractor/crunchyroll] Better message for premium videos
Closes #6227
2023-02-14 01:07:07 +05:30
MarengaandGitHub a9c685453f [extractor/vk] Fix playlists for new API (#6122)
Authored by: the-marenga
Closes #6219
2023-02-13 11:37:47 +05:30
pukkandan c154302c58 Bugfix for 39f32f1715 2023-02-13 01:35:54 +05:30
pukkandan 5712943b76 Imply --no-progress when --print 2023-02-13 01:19:51 +05:30
pukkandan 39f32f1715 Sanitize formats before sorting
Closes #4501
2023-02-13 01:19:51 +05:30
shirtandGitHub 365b900605 [Build] Update pyinstaller 2023-02-12 10:57:57 -05:00
nixxoandGitHub c6b657867a [extractor/rcs] Fix extractors (#5700)
Authored by: nixxo, pukkandan
Closes #5683
2023-02-12 20:13:20 +05:30
LesmiscoreandGitHub a4f1683221 [extractor/AbemaTV] Cache user token whenever appropriate (#6216)
Authored by: Lesmiscore
2023-02-12 23:02:09 +09:00
Simon SawickiandGitHub b6795fd310 [extractor/twitter] Fix --no-playlist and add media view_count when using GraphQL (#6211)
Authored by: Grub4K
2023-02-12 14:43:26 +01:00
pukkandan 2e269bd998 [pyinst] Fix for pyinstaller 5.8
Fixes comment https://github.com/yt-dlp/yt-dlp/issues/1839#issuecomment-1427002271
2023-02-12 18:43:21 +05:30
Bruno GuerreiroandGitHub 78a78fa74d [extractor/youtube] Add hyperpipe instances (#6020)
Authored by: Generator
2023-02-12 14:03:45 +05:30
HobbyistDevandGitHub 0ba87dd279 [extractor/biliintl] Add intro and ending chapters (#6018)
Authored by: HobbyistDev
2023-02-12 13:24:36 +05:30
Roland Hieberandpukkandan 05799a48c7 [extractor/youtube] Update invidious and piped instances (#6030)
Authored by: rohieb
2023-02-12 13:22:07 +05:30
ByteDreamandGitHub 93abb7406b [extractor/crunchyroll] Add intro chapter (#6023)
Authored by: ByteDream
2023-02-12 13:17:12 +05:30
LowSuggestion912andGitHub b23167e754 [extractor/common] Fix _search_nuxt_data (#6062)
Authored by: LowSuggestion912
2023-02-12 12:55:24 +05:30
Chris CarusoandGitHub 417cdaae08 [extractor/ximalaya] Update album _VALID_URL (#6110)
Authored by: carusocr
Closes #6059
2023-02-12 10:23:24 +05:30
seproandGitHub b3eaab7ca2 [extractor/vlive] Replace with VLiveWebArchiveIE (#6196)
vlive has shut down: https://web.archive.org/web/20221031171019/https://www.vlive.tv/notice/4749

Authored by: seproDev
2023-02-12 10:17:03 +05:30
lauren n. liberdaandGitHub a31d0fa6c3 [extractor/tvp] Support stream.tvp.pl (#6139)
Authored by: selfisekai
2023-02-12 10:13:10 +05:30
seproandGitHub cc2389c8ac [extractor/npo] Fix extractor and add HD support (#6155)
Authored by: seproDev
2023-02-12 10:05:24 +05:30
Chris CarusoandGitHub 20266508dd [extractor/bfmtv] Support rmc prefix (#6025)
Authored by: carusocr
Closes #6021
2023-02-12 09:59:41 +05:30
qulazandGitHub cc13293c28 [extractor/clyp] Support wav (#6102)
Authored by: qulaz
2023-02-12 09:58:15 +05:30
oxamunandGitHub 989f47b631 [extractor/tnaflix] Fix extractor (#6086)
Closes #6085
Authored by: oxamun, bashonly
2023-02-12 09:51:29 +05:30
JChris246andGitHub 7d5f919bad [extractor/Stripchat] Fix extractor (#5985)
Authored by bashonly, JChris246
Closes #5963, closes #5866
2023-02-12 09:47:37 +05:30
panatexxaandGitHub c62e64cf01 [extractor/moviepilot] Fix extractor (#5954)
Authored by: panatexxa
2023-02-12 09:45:16 +05:30
pmitchell86andGitHub c085cc2def [extractor/91porn] Fix title and comment extraction (#5932)
Authored by: pmitchell86
Fixes #3256
2023-02-12 09:43:31 +05:30
Alex BergandGitHub 7708df8da0 [extractor/Hidive] Fix subtitles and age-restriction (#5828)
Authored by: chexxor
Closes #408
2023-02-12 09:17:52 +05:30
pukkandan b85faf6ffb [devscripts/pyinstaller] Analyze sub-modules of Cryptodome
Ref: https://github.com/yt-dlp/yt-dlp/issues/6185#issuecomment-1423523986
2023-02-12 03:07:32 +05:30
MasterandGitHub 203a06f855 [extractor/radiko] Fix format sorting for Time Free (#6159)
Authored by: road-master
2023-02-11 19:24:10 +09:00
Simon Sawickiandpukkandan 6839ae1f6d [utils] traverse_obj: Fix more bugs
and cleanup uses of `default=[]`

Continued from b1bde57bef
2023-02-10 19:36:55 +05:30
LeoniePhilineandGitHub c0cd13fb1c [extractor/vimeo] Fix playerConfig extraction (#6203)
Authored by: bashonly, LeoniePhiline
Closes #6149
2023-02-10 19:20:29 +05:30
Ha Tien LoiandGitHub f14c233348 [extractor/DouyuTV]: Use new API (#6074)
Authored by: hatienl0i261299
2023-02-09 02:11:04 +05:30
pukkandan 768a001781 [compat_utils] Simplify EnhancedModule 2023-02-09 01:47:13 +05:30
pukkandan acb1042a9f [devscripts] Provide pyinstaller hooks
Closes #6185
2023-02-09 01:46:56 +05:30
f40e32fb1a [extractor/servus] Rewrite extractor (#6036)
Closes #1076, closes #4240, closes #2748, closes #1045, closes #1498
Authored by: FrankZ85, Ashish0804, StefanLobbenmeier

Co-authored-by: FrankZ85 <43293037+FrankZ85@users.noreply.github.com>
2023-02-08 11:35:32 +05:30
e61acb40b2 [extractor/wrestleuniverse] Add extractors (#6158)
Authored by bashonly, Grub4K
Closes #6120

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
2023-02-08 11:12:11 +05:30
7e68567e50 [downloader/hls] Allow extractors to provide AES key (#6158)
and related cleanup

Authored by: bashonly, Grub4K

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
2023-02-08 11:09:32 +05:30
JChris246andGitHub f7efe6dc95 [extractor/pornez] Handle relative URLs in iframe (#6171)
Authored by: JChris246
Closes #6162
2023-02-08 10:50:19 +05:30
Simon SawickiandGitHub b1bde57bef [utils] traverse_obj: Fix several behavioral problems
See #6180 for further info

Authored by: Grub4K
2023-02-08 04:11:08 +01:00
pukkandan 88426d9446 [compat_utils] Improve passthrough_module 2023-02-08 08:23:36 +05:30
pukkandan f6a765ceb5 [dependencies] Standardize Cryptodome imports 2023-02-08 07:28:46 +05:30
pukkandan 754c84e2e4 Support module level __bool__ and property 2023-02-08 07:28:45 +05:30
pukkandan 7aefd19afe Make title completely non-fatal
Ref: https://github.com/yt-dlp/yt-dlp/pull/6158#discussion_r1096984349
2023-02-07 01:18:04 +05:30
Felix YanandGitHub fbbb5508ea [extractor/huya] Support HD streams (#6172)
Authored by: felixonmars
2023-02-07 00:54:47 +05:30
OMEGA_RAZERandGitHub c77df98b1a [extractor/reddit] Support user posts (#6173)
Authored by: OMEGARAZER
2023-02-06 19:21:39 +05:30
Jeroen JacobsandGitHub d27bde9883 [extractor/GoPlay] Use new API (#6151)
Authored by: jeroenj
Closes #6032
2023-02-04 04:12:43 +05:30
seproandGitHub 0fe87a8730 [extractor/zdf] Use android API endpoint for UHD downloads (#6150)
Authored by: seproDev
2023-02-04 04:08:29 +05:30
MatumoandGitHub 3b161265ad [extractor/niconico] Add support for like history (#5705)
Authored by: Matumo, pukkandan
2023-02-04 00:20:06 +05:30
chio0haiandGitHub 389896df85 [extractor/txxx] Add extractors (#5240)
Authored by: chio0hai
Closes #5021
2023-02-04 00:17:00 +05:30
pukkandan b032ff0f03 [extractor/youtube] Handle consent.youtube 2023-02-03 23:53:42 +05:30
pukkandan dad2210c0c [extractor/youtube] Support /live/ URL 2023-02-03 23:53:41 +05:30
Jasper RebaneandGitHub 9cfdbcbf3f [extractor/freesound] Workaround invalid URL in webpage (#6147)
Authored by: rebane2001
Closes #6146
2023-02-03 20:08:51 +05:30
lauren n. liberdaandGitHub 7543c9c99b [extractor/twitter] Fix graphql extraction on some tweets (#6075)
Authored by: selfisekai
2023-02-02 19:02:14 +05:30
Simon Sawicki acacb57c7e [extractor/rumble] Fix format sorting
Closes #6119
Authored by: pukkandan
2023-02-02 07:12:36 +01:00
Simon SawickiandGitHub 776995bc10 [utils] traverse_obj: Various improvements
- Add `set` key for transformations/filters
- Add `re.Match` group names
- Fix behavior for `expected_type` with `dict` key
- Raise for filter function signature mismatch in debug

Authored by: Grub4K
2023-02-02 06:40:19 +01:00
pukkandan 8b008d6254 [jsinterp] Support if statements
Closes #6131
2023-02-01 09:40:16 +05:30
LesmiscoreandGitHub 83c4970e52 [utils] Fix time_seconds to use the provided TZ (#6118)
Authored by: Lesmiscore, Grub4K

Fixes https://github.com/yt-dlp/yt-dlp/pull/6056
2023-01-31 22:30:00 +09:00
bashonly 8aa0bd5d10 [extractor/generic] Avoid catastrophic backtracking in KVS regex
Authored by: bashonly
2023-01-29 00:59:37 -06:00
Simon Sawicki 37e325b92f [utils] Use local kernel32 for file locking on Windows
Ref: https://github.com/ytdl-org/youtube-dl/issues/21545

Authored by: Grub4K
2023-01-25 22:32:07 +01:00
pukkandan 59d7de0da5 Fix --concat-playlist
Closes #6080
2023-01-24 03:43:48 +05:30
pukkandan 88d8928bf7 [plugins] Fix zip search paths
Closes #6011
2023-01-20 23:35:34 +05:30
bashonly 176a068cde [extractor/nbc] Fix XML parsing
Python 3.7 compat bug in cb73b8460c
Authored by: bashonly
2023-01-16 15:38:33 -06:00
bashonlyandGitHub 5ab3534d44 [extractor/slideslive] Fix slides and chapters/duration (#6024)
* Fix slides/thumbnails extraction
* Extract duration to fix issues w/ `--embed-chapters`, `--split-chapters`
* Add `InfoExtractor._extract_mpd_vod_duration` method
* Expand applicability of `InfoExtractor._parse_m3u8_vod_duration` method
Authored by: bashonly
2023-01-14 19:52:03 +00:00
bashonlyandGitHub cb73b8460c [extractor/nbc] Fix NBC and NBCStations extractors (#6033)
Improve `InfoExtractor._parse_smil_formats` extension detection
Closes #6019
Authored by: bashonly
2023-01-14 16:40:42 +00:00
bashonlyandGitHub 7481998b16 [extractor/drtv] Fix bug in ab4cbef (#6034)
Fixes bug in ab4cbef ab4cbeff00
Closes #5993
Authored by: bashonly
2023-01-14 16:35:47 +00:00
pukkandan 87ebab0615 [extractor/embedly] Embedded links may be for other extractors
Bug in bfd973ece3
Closes #5987
2023-01-08 00:39:12 +05:30
Marek HudikandGitHub 355d781bed [extractor/rozhlas] Add extractor RozhlasVltavaIE (#5951)
Authored by: amra
2023-01-07 20:37:10 +05:30
github-actions 7287ab92f6 [version] update
Created by: pukkandan

:ci skip all :ci run dl
2023-01-06 21:21:26 +00:00
pukkandan 6becd2508c Release 2023.01.06 2023-01-07 02:48:35 +05:30
pukkandan edfc7725b1 [cleanup] Misc 2023-01-07 02:48:34 +05:30
JChris246andGitHub b382c1fc6a [xanimu] Add extractor (#5969)
Authored by: JChris246
Closes #5810
2023-01-07 01:39:37 +05:30
Christoph FlathmannandGitHub 8a6b167723 [extractor/crunchyroll:show] Add language to entries (#5687)
Authored by: Chrissi2812
2023-01-07 01:05:03 +05:30
mzhouandGitHub 253ac4ba6a [extractor/youtube] Retry manifest refresh for live-from-start (#5670)
Avoids ending download early when live stream is temporarily offline.
Best used with somewhat large `--retry-sleep extractor:` and `--extractor-retries`

Authored by: mzhou
2023-01-07 01:00:42 +05:30
George SchizasandGitHub 84e0e33a19 [extractor/reddit] Add subreddit as channel_id (#5685)
Authored by: gschizas
Closes #5684
2023-01-07 00:57:02 +05:30
Frederik Nordahl Jul SabroeandGitHub ab4cbeff00 [extractor/drtv] Add series extractors (#5644)
Authored by: FrederikNS
Closes #3567
2023-01-07 00:37:52 +05:30
Simon SawickiandGitHub 773c272d66 Fix config locations (#5933)
Bug in 8e40b9d1ec
Closes #5953

Authored by: Grub4k, coletdjnz, pukkandan
2023-01-07 00:31:00 +05:30
Jacob TrumanandGitHub c3366fdfd0 [extractor/nbc] Update graphql query (#5952)
Closes #5918
Authored by: jacobtruman
2023-01-07 00:14:35 +05:30
Simon SawickiandGitHub 5be214abed [update] Fix updater file removal on windows (#5970)
Reverts 2fb0f85868
Closes #5632
Authored by: Grub4K
2023-01-06 22:31:18 +05:30
HobbyistDevandGitHub d37422f1db [extractor/biliIntl] Add fallback to video_data (#5971)
Authored by: HobbyistDev
2023-01-06 11:52:25 +05:30
JC-ChungandGitHub 933ed882e9 [extractor/tiktok] Add TikTokLive extractor (#5637)
Closes #3698
Authored by: JC-Chung
2023-01-05 11:23:34 +00:00
HobbyistDevandGitHub a1d9aca338 [extractor/aitube] Add extractor (#5946)
Closes #5627
Authored by: HobbyistDev
2023-01-04 17:03:36 +05:30
HobbyistDevandGitHub 91d54e9b99 [extractor/volejtv] Add extractor (#5943)
Authored by: HobbyistDev
Closes #5883
2023-01-04 13:20:23 +05:30
HobbyistDevandGitHub 76c3ceccfb [extractor/biliintl] Add /media to VALID_URL (#5939)
Authored by: HobbyistDev
2023-01-03 23:29:52 +05:30
pukkandan ad68b16a1e [downloader/aria2c] Disable native progress
Closes #5931, closes #5928, Re-opens #2038
2023-01-03 17:25:56 +05:30
pukkandan f079514957 [utils] windows_enable_vt_mode: Better error handling
Closes #5927
2023-01-03 15:59:49 +05:30
pukkandan e9df3d42c4 [build] Add minimal pyproject.toml 2023-01-03 11:25:01 +05:30
pukkandan d80ca5deaa [utils] mimetype2ext: weba is not standard
Fix bug in fbb7383306, 2647c933b8
Closes #5935
2023-01-03 11:25:01 +05:30
OndrejBakanandGitHub 1a3cd8ec35 [extractor/joj] Fix extractor (#5934)
Authored by: OndrejBakan, pukkandan
2023-01-03 11:05:05 +05:30
github-actions 990dd7b00f [version] update
Created by: pukkandan

:ci skip all :ci run dl
2023-01-02 14:44:06 +00:00
pukkandan d83b0ad809 Release 2023.01.02 2023-01-02 20:07:07 +05:30
pukkandan 08e29b9f1f [cleanup] Misc
Closes #5576, closes #5887
2023-01-02 19:40:15 +05:30
pukkandan 8e174ba7de [docs] Improvements
Closes #5846, closes #5774
2023-01-02 19:40:13 +05:30
bashonlyandGitHub 05997b6e98 [extractor/generic] Decode unicode-escaped embed URLs (#5919)
Authored by: bashonly
Closes #5854
2023-01-02 19:36:01 +05:30
Simon SawickiandGitHub 32a84bcf4e Update to ytdl-commit-195f22f6
[generic] Improve KVS (etc) extraction
https://github.com/ytdl-org/youtube-dl/commit/195f22f679330549882a8234e7234942893a4902

Closes #3716
Authored by: Grub4k, pukkandan
2023-01-02 19:15:36 +05:30
MatthewandGitHub 8300774c4a Add --enable-file-urls (#5917)
Closes https://github.com/yt-dlp/yt-dlp/issues/3675

Authored by: coletdjnz
2023-01-02 06:05:13 +00:00
bashonlyandGitHub d7f9871469 [extractor/iqiyi] Fix Iq JS regex (#5922)
Closes #5702
Authored by: bashonly
2023-01-02 05:50:37 +00:00
bashonlyandGitHub 13f930abc0 [extractor/fifa] Fix Preplay extraction (#5921)
Closes #5839
Authored by: dirkf
2023-01-02 05:46:06 +00:00
bashonlyandGitHub b23b503e22 [extractor/odnoklassniki] Extract subtitles (#5920)
Closes #5744
Authored by: bashonly
2023-01-02 05:44:54 +00:00
MatthewandGitHub e756f45ba0 Improve handling for overriding extractors with plugins (#5916)
* Extractors replaced with plugin extractors now show in debug output
* Better testcase handling
* Added documentation
Authored by: coletdjnz, pukkandan
2023-01-02 04:55:11 +00:00
LesmiscoreandGitHub 8c53322cda [downloader/aria2c] Native progress for aria2c via RPC (#3724)
Authored by: Lesmiscore, pukkandan

Closes #2038
2023-01-02 02:16:25 +09:00
pukkandan 193fb150b7 Fix bug in 119e40ef64 2023-01-01 17:01:48 +05:30
pukkandan 26fdfc3704 [extractor/biliintl:series] Make partial download of series faster 2023-01-01 14:41:47 +05:30
pukkandan 78d25e0b7c [extractor/embedly] Handle vimeo embeds
Closes #3360
2023-01-01 14:15:23 +05:30
pukkandan 2a06bb4eb6 Add --compat-options 2021,2022
Use these to guard against future compat changes. This allows devs to
change defaults and make other potentially breaking changes more easily.
If you need everything to work exactly as-is, put this in your config
2023-01-01 14:11:15 +05:30
pukkandan 88fb942577 Add message when there are no subtitles/thumbnails
Closes #5551
2023-01-01 14:11:15 +05:30
pukkandan 1cdda32998 [utils] get_exe_version: Detect broken executables
Authored by: dirkf, pukkandan
Closes #5561
2023-01-01 14:11:14 +05:30
coletdjnz 3e01ce744a [extractor/generic] Use Accept-Encoding: identity for initial request
The existing comment seems to imply this was the desired behavior from the beginning.

Partial fix for https://github.com/yt-dlp/yt-dlp/issues/5855, https://github.com/yt-dlp/yt-dlp/issues/5851, https://github.com/yt-dlp/yt-dlp/issues/4748
2023-01-01 18:41:35 +13:00
MatthewandGitHub 8e40b9d1ec Improve plugin architecture (#5553)
to make plugins easier to develop and use:
* Plugins are now loaded as namespace packages.
* Plugins can be loaded in any distribution of yt-dlp (binary, pip, source, etc.).
* Plugin packages can be installed and managed via pip, or dropped into any of the documented locations.
* Users do not need to edit any code files to install plugins.
* Backwards-compatible with previous plugin architecture.

As a side-effect, yt-dlp will now search in a few more locations for config files.

Closes https://github.com/yt-dlp/yt-dlp/issues/1389

Authored by: flashdagger, coletdjnz, pukkandan, Grub4K
Co-authored-by: Marcel <flashdagger@googlemail.com>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
Co-authored-by: Simon Sawicki <accounts@grub4k.xyz>
2023-01-01 04:29:22 +00:00
pukkandan 2fb0f85868 [update] Workaround #5632 2022-12-31 11:04:02 +05:30
Stel AbregoandGitHub a0e526ed4d [extractor/bandcamp] Add album_artist (#5537)
Closes #5536 
Authored by: stelcodes
2022-12-31 10:28:33 +05:30
pukkandan 8d1ddb0805 [extractor/udemy] Fix lectures that have no URL and detect DRM
Closes #5662
2022-12-31 10:02:50 +05:30
pukkandan 9bb856998b [extractor/youtube] Extract DRC formats 2022-12-30 15:50:17 +05:30
pukkandan fbb7383306 Add weba to known extensions 2022-12-30 15:32:47 +05:30
pukkandan ec54bd43f3 Fix bug in writing playlist info-json
Closes #4889
2022-12-30 14:07:15 +05:30
pukkandan f74371a97d [extractor/bilibili] Fix --no-playlist for anthology
Closes #5797
2022-12-30 12:10:57 +05:30
ChillingPepperandGitHub d5f043d127 [utils] js_to_json: Fix bug in f55523c (#5771)
Authored by: ChillingPepper, pukkandan
2022-12-30 12:08:38 +05:30
pukkandan fe74d5b592 Let --parse/replace-in-metadata run at any post-processing stage
Closes #5808, #456
2022-12-30 11:19:39 +05:30
pukkandan 119e40ef64 Add pre-processor stage video
Related: #456, #5808
2022-12-30 11:18:45 +05:30
pukkandan 4455918e7f [extractor/stv] Detect DRM
Closes #5320
2022-12-30 11:17:10 +05:30
Anant MurmuandGitHub efa944f4bc [cleanup] Use random.choices (#5800)
Authored by: freezboltz
2022-12-30 08:13:49 +05:30
nosoopandGitHub e107c2b8cf [extractor/soundcloud] Support user permalink (#5842)
Closes #5841
Authored by: nosoop
2022-12-30 00:16:43 +05:30
Lesmiscore ca2f6e14e6 [extractor/BiliLive] Fix extractor
- Remove unnecessary group in `_VALID_URL`
- This extractor always returns livestreams
2022-12-30 03:01:22 +09:00
bashonlyandGitHub c1edb853b0 [extractor/kick] Add extractor (#5736)
Closes #5722
Authored by: bashonly
2022-12-29 17:31:01 +00:00
bashonlyandGitHub 2647c933b8 [extractor/wistia] Improve extension detection (#5415)
Closes #5053
Authored by: bashonly, Grub4k, pukkandan
2022-12-29 16:32:54 +00:00
bashonlyandGitHub 53006b35ea [extractor/amazon] Add AmazonReviews extractor (#5857)
Closes #5766
Authored by: bashonly
2022-12-29 15:04:09 +00:00
bashonlyandGitHub 4b183d4962 [extractor/videoken] Add extractors (#5824)
Closes #5818
Authored by: bashonly
2022-12-29 14:29:08 +00:00
bashonlyandGitHub 3d667e0047 [extractor/slideslive] Support embeds and slides (#5784)
Authored by: bashonly, Grub4K, pukkandan
2022-12-29 12:03:03 +00:00
SamandGitHub 9a9006ba20 [extractor/twitcasting] Fix videos with password (#5894)
Closes #5888
Authored by: bashonly, Spicadox
2022-12-29 11:15:38 +00:00
HobbyistDevandGitHub 153e88a751 [extractor/netverse] Add NetverseSearch extractor (#5838)
Authored by: HobbyistDev
2022-12-29 13:42:07 +05:30
JChris246andGitHub 9fcd8ad1f2 [extractor/spankbang] Fix extractor (#5791)
Authored by: JChris246
Closes #5731
2022-12-29 13:38:22 +05:30
monnefandGitHub 6b71d186dd [extractor/curiositystream] Fix auth (#5730)
Authored by: mnn
2022-12-29 13:17:23 +05:30
lkw123andGitHub 074b2fae90 [extractor/kankanews] Add extractor (#5729)
Authored by: synthpop123
2022-12-29 13:08:49 +05:30
Kurt BestorandGitHub 06a9d68eb8 [extractor/youku] Fix extractor (#5622)
Closes #4456
Authored by: KurtBestor
2022-12-29 12:48:55 +05:30
Damiano AmatrudaandGitHub a4d6ead30f [extractor/ciscowebex] Support password-protected videos (#5601)
Authored by: damianoamatruda
2022-12-29 12:24:19 +05:30
lauren n. liberdaandGitHub d1b5f3d79c [extractor/polskieradio] Adapt to next.js redesigns (#5416)
Authored by: selfisekai
2022-12-28 02:17:25 +05:30
lauren n. liberdaandGitHub da8d2de208 [extractor/cda] Support premium and misc improvements (#5529)
* Fix cache for non-ASCII key
* Improve error messages
* Better UA for fingerprint bypass

Authored by: selfisekai
2022-12-28 01:27:26 +05:30
chrisandGitHub 15e9e578c0 [extractor/ArteTV] Extract chapters (#5879)
Authored by: iw0nderhow, bashonly
2022-12-28 01:22:58 +05:30
BobscornandGitHub 0ef3d47027 [extractor/beatbump] Add extractors (#5304)
Authored by: Bobscorn, pukkandan
Closes #4653
2022-12-27 12:34:56 +05:30
barsnickandGitHub 247c8dd4f5 [extractor/urplay] Support for audio-only formats (#4606)
Closes #4605
Authored by: barsnick
2022-12-27 12:04:01 +05:30
HobbyistDevandGitHub 032f22020c [extractor/trtcocuk] Add extractor (#5009)
Closes #2635
Authored by: HobbyistDev
2022-12-27 11:55:09 +05:30
pukkandan 4af47a0003 Fix 9012d20b23 2022-12-27 11:45:22 +05:30
pukkandan 9012d20b23 [extractor/mixch] Support --wait-for-video 2022-12-27 03:01:22 +05:30
Giulio MuscarelloandGitHub d61ef7f343 [extractor/ARD] Add vtt subtitles (#5835)
Authored by: CapacitorSet
2022-12-24 16:19:10 +05:30
skbehandGitHub 1c226ccdd4 [extractor/bilibili] Improve _VALID_URL (#5820)
Authored by: skbeh
2022-12-24 16:17:37 +05:30
pukkandan 8791e78ccc Fix original_url in playlists 2022-12-23 01:44:20 +05:30
pukkandan 69f5fe45b9 [FFmpegVideoConvertor] Add gif to --recode-video 2022-12-23 01:44:20 +05:30
pukkandan 0b5546c723 [extractor] Let _extract_format functions obey --ignore-no-formats 2022-12-23 01:44:18 +05:30
bashonlyandGitHub 1fc089143c [extractor/reddit] Extract crossposted media (#5801)
Closes #5798
Authored by: bashonly
2022-12-21 00:55:47 +00:00
LesmiscoreandGitHub 5424dbaf91 Deprioritize HEVC-over-FLV formats (#5823)
Authored by: Lesmiscore
2022-12-19 11:36:14 +09:00
MatthewandGitHub c733555106 [extractor/youtube:tab] Extract metadata from channel items (#5569)
Authored by: coletdjnz
2022-12-12 23:08:14 +00:00
HobbyistDevandGitHub 81388c0954 [extractor/oneplace] Add OnePlacePodcast extractor (#5549)
Closes #5543
Authored by: HobbyistDev
2022-12-10 19:10:24 +05:30
DenisandGitHub df10bad267 [extractor/rutube] Support private videos (#5761)
Authored by: mexus
2022-12-10 18:47:01 +05:30
HobbyistDevandGitHub f0f3fa028b [extractor/netverse] Extract comments (#5568)
Authored by: HobbyistDev
2022-12-10 14:17:06 +05:30
HobbyistDevandGitHub 22697a84f6 [extractor/europarl] Add EuroParlWebstream Extractor (#5547)
Authored by: HobbyistDev
Closes #4933
2022-12-10 14:14:43 +05:30
HobbyistDevandGitHub 3ac5476430 [extractor/nosnl] Add support for /video (#5590)
Authored by: HobbyistDev
2022-12-10 14:04:55 +05:30
HobbyistDevandGitHub e318b5b87a [extractor/airtv] Add extractor (#5533)
Authored by: HobbyistDev
Closes #5132
2022-12-10 13:59:13 +05:30
bashonlyandGitHub f549b18512 [extractor/pinterest] Fix extractor (#5739)
Closes #1772
Authored by: bashonly
2022-12-09 23:46:04 +00:00
bashonlyandGitHub 7c5e1701f6 [extractor/foxsports] Fix extractor (#5719)
Closes #5714
Authored by: bashonly
2022-12-09 23:43:10 +00:00
bashonlyandGitHub 16bed382fd [extractor/twitter] Heed --no-playlist for multi-video tweets (#5757)
Closes #5752
Authored by: bashonly, Grub4K
2022-12-09 23:41:45 +00:00
bashonlyandGitHub 3cf50fa8e9 [downloader/ffmpeg] Fix headers for video+audio formats (#5659)
Authored by: bashonly, Grub4K
2022-12-09 23:36:38 +00:00
bashonlyandGitHub f69b0554eb [extractor/slideslive] Fix extractor (#5737)
Closes #1532
Authored by: bashonly, Grub4K
2022-12-09 23:25:37 +00:00
pukkandan e74a3c6dcc [extractor/hotstar] Improve format metadata 2022-12-09 15:23:59 +05:30
pukkandan 7108221662 Add ac4 to known codecs
Note: ffmpeg does not currently support this format

Related #5738
2022-12-09 15:23:59 +05:30
nixxoandGitHub 10dc85924a [extractor/mediaset] Better embed detection and error messages (#5664)
Authored by: nixxo
2022-12-09 12:50:37 +05:30
VitaandGitHub b05f0a50e0 [extractor/yle_areena] Support restricted videos (#5735)
* and improve metadata

Closes #5734
Authored by: docbender
2022-12-09 11:33:36 +05:30
3d79ebc8b7 [extractor/mediastream] Add extractor (#5640)
Closes #5532, closes #4431, closes #4425
Authored by: elyse0, HobbyistDev

Co-authored-by: HobbyistDev <tesutonihon4@gmail.com>
2022-12-09 02:47:21 +05:30
pukkandan b44cd29851 [jsinterp] Escape regex that looks like nested set
Closes #5749
2022-12-08 22:43:38 +05:30
milkknifeandGitHub 85a802969e [extractor/webcamerapl] Add extractor (#5715)
Authored by: milkknife
2022-12-08 22:26:36 +05:30
nixxoandGitHub 72f96c5566 [extractor/la7] Improve extractor (#5538)
Authored by: nixxo
Closes #5360
2022-12-08 22:22:19 +05:30
MMMandGitHub 839e2a62ae [extractor/rumble] Add RumbleIE extractor (#5515)
Closes #2846
Authored by: flashdagger
2022-12-08 22:02:17 +05:30
HobbyistDevandGitHub 28b8f57b4b [extractor/noice] Add NoicePodcast extractor (#5621)
Authored by: HobbyistDev
2022-12-08 19:28:36 +05:30
lkw123andGitHub dfc186d422 [extractor/xiami] Remove extractors (#5711)
Authored by: synthpop123
2022-12-08 18:13:29 +05:30
David TurnerandGitHub 42ec478fc4 [extractor/plutotv] Fix videos with non-zero start (#5745)
Authored by: digitall
2022-12-08 18:08:52 +05:30
pukkandan 7991ae57a8 [extractor/sibnet] Separate from VKIE
Fixes https://github.com/yt-dlp/yt-dlp/commit/bfd973ece3369c593b5e82a88cc16de80088a73e#commitcomment-91834251
2022-12-08 17:20:02 +05:30
pukkandan 935bac1e4d Fix --cookies-from-browser CLI parsing
Closes #5716
2022-12-06 00:35:53 +05:30
bashonlyandGitHub c4cbd3bebd [extractor/tiktok] Update _VALID_URL, add api_hostname arg (#5708)
Closes #5706
Authored by: bashonly
2022-12-04 22:30:31 +00:00
pukkandan c53a18f016 [utils] windows_enable_vt_mode: Proper implementation
Authored by: Grub4K
2022-12-05 01:06:56 +05:30
pukkandan 71df9b7fd5 [cleanup] Misc 2022-12-03 19:52:31 +05:30
Benjamin RyanandGitHub c9f5ce5118 [extractor/tiktok] Update API hostname (#5690)
Closes #5688
Authored by: redraskal
2022-12-02 09:38:00 +00:00
bashonlyandGitHub ddf1e22d48 [extractor/swearnet] Fix description bug (#5681)
Bug in 049565df2e
Closes #5643
Authoried by: bashonly
2022-12-01 11:24:43 +00:00
bashonlyandGitHub 0e96b408b9 [extractor/reddit] Extract video embeds in text posts (#5677)
Closes #5612
Authored by: bashonly
2022-12-01 04:04:32 +00:00
bashonlyandGitHub ba72399723 [extractor/tiktok] Fix subs, DouyinIE, improve _VALID_URL (#5676)
Closes #5665, Closes #2267
Authored by: bashonly
2022-12-01 04:00:32 +00:00
pukkandan 9bcfe33be7 [utils] Make ExtractorError mutable 2022-11-30 06:10:26 +05:30
pukkandan 71eb82d1b2 [extractor/youtube] Subtitles cannot be translated to und
Closes #5674
2022-11-30 05:18:18 +05:30
pukkandan a9d069f5b8 [extractor/amazonminitv] Cleanup 48652590ec 2022-11-29 07:54:07 +05:30
alexiaandGitHub 48652590ec [extractor/amazonminitv] Add extractors (#3628)
Authored by: nyuszika7h, GautamMKGarg
2022-11-28 11:36:18 +09:00
marieellandGitHub 86f557b636 [extractor/youporn] Fix metadata (#2768)
Authored by: marieell
2022-11-26 08:00:25 +05:30
pukkandan c0caa80515 [extractor/naver] Treat fan subtitles as separate language
Closes #5467
2022-11-25 16:10:30 +05:30
Mudassir ChapraandGitHub 0d95d8b00a [extractor/gronkh] Fix _VALID_URL (#5628)
Closes #5531
Authored by: muddi900
2022-11-24 15:34:45 +00:00
Elan RuusamäeandGitHub 9d52bf65ff [extractor/kanal2] Add extractor (#5575)
Authored by: glensc, pukkandan, bashonly
2022-11-22 18:09:57 +00:00
bashonlyandGitHub d761dfd059 [extractor/naver] Improve _VALID_URL for NaverNowIE (#5620)
Authored by: bashonly
2022-11-22 03:42:16 +00:00
bashonlyandGitHub 27c0f899c8 [extractor/screencastify] Add extractor (#5604)
Closes #5603
Authored by: bashonly
2022-11-22 00:40:02 +00:00
bashonlyandGitHub 7ff2fafe47 [extractor/vimeo] Add VimeoProIE (#5596)
* Add support for VimeoPro URLs not containing a Vimeo video ID
* Add support for password-protected VimeoPro pages
Closes #5594
Authored by: bashonly, pukkandan
2022-11-21 00:55:57 +00:00
bashonlyandGitHub 3b021eacef [extractor/generic] Add fragment_query extractor arg for DASH and HLS (#5528)
* `fragment_query`: passthrough any query in generic mpd/m3u8 manifest URLs to their fragments
* Add support for `extra_param_to_segment_url` to DASH downloader
Authored by: bashonly, pukkandan
2022-11-21 00:51:45 +00:00
Marcelandpukkandan f352a09778 [webvtt] Handle premature EOF
Closes #2867, closes #5600
Authored by: flashdagger
2022-11-20 14:14:42 +05:30
chengzhicnandGitHub 02b2f9fa7d [extractor/reddit] Add vcodec to fallback format (#5591)
Authored by: chengzhicn
2022-11-20 01:44:21 +05:30
pukkandan 29ca408219 [FormatSort] Add mov to vext
Closes #5581
2022-11-19 09:04:01 +05:30
pukkandan 8486540257 [extractor/unsupported] Add more URLs
Closes #5557, Closes #2744, Closes #5578
2022-11-19 08:42:06 +05:30
bashonlyandGitHub ed027fd9d8 [extractor/generic] Fix JSON LD manifest extraction (#5577)
Closes #5572
Authored by: bashonly, pukkandan
2022-11-18 02:04:03 +00:00
bashonlyandGitHub 352e7d9873 [extractor/twitter] Refresh guest token when expired (#5560)
Closes #5548
Authored by: bashonly, Grub4K
2022-11-18 02:00:11 +00:00
nixxoandGitHub 9a0416c6a5 [extractor/twitter:spaces] Add 'Referer' to m3u8 (#5580)
Closes #5565
Authored by: nixxo
2022-11-18 06:42:02 +05:30
bashonlyandGitHub f5a9e9df0d [extractor/brightcove] Add BrightcoveNewBaseIE and fix embed extraction (#5558)
* Move Brightcove embed extraction and tests into the IEs
* Split `BrightcoveNewBaseIE` from `BrightcoveNewIE`
* Fix bug in ade1fa70cb with the "wrong" spelling of `referrer` being smuggled

Closes #5539
2022-11-17 19:11:35 +00:00
bashonlyandGitHub f96a3fb7d3 [extractor/redgifs] Fix bug in 8c188d5d09 (#5559) 2022-11-17 19:09:40 +00:00
BnyroandGitHub bc87dac75f [extractor/youtube] Add piped.video (#5571)
Closes #5518
Authored by: Bnyro
2022-11-17 18:45:38 +05:30
pukkandan 9f14daf22b [extractor] Deprecate _sort_formats 2022-11-17 11:40:17 +05:30
pukkandan 784320c98c Implement universal format sorting
Closes #5566
2022-11-17 11:05:49 +05:30
pukkandan d0d74b7197 [utils] Move format sorting code into utils 2022-11-17 11:04:38 +05:30
pukkandan 64c464a144 [utils] Move FileDownloader.parse_bytes into utils 2022-11-17 08:40:34 +05:30
pukkandan 4de88a6a36 [extractor/generic] Don't report redirect to https 2022-11-17 02:12:07 +05:30
pukkandan 105bfd90f5 Add new field aspect_ratio
Closes #5402
2022-11-16 06:57:09 +05:30
pukkandan 6368e2e639 [cleanup] Misc
Closes #5541
2022-11-16 06:57:07 +05:30
pukkandan a4894d3e25 [extractor/youtube] Consider language in format de-duplication 2022-11-15 05:23:46 +05:30
pukkandan d7b460d0e5 Make early reject of --match-filter stricter
Closes #5509
2022-11-13 10:56:06 +05:30
pukkandan 171a31dbe8 [extractor] Add a way to distinguish IEs that returns only videos 2022-11-13 10:56:04 +05:30
pukkandan 83cc7b8aae [utils] classproperty: Add cache support 2022-11-13 08:29:49 +05:30
ElyseandGitHub 0a4b2f4180 [extractor/tencent] Fix geo-restricted video (#5505)
Closes #5230
Authored by: elyse0
2022-11-12 12:43:13 +05:30
pukkandan a8c754cc00 [extractor/youtube] Fix bug in handling of music URLs
Bug in bd7e919a75
Closes #5502
2022-11-12 00:02:13 +05:30
pukkandan bc5c2f8a2c Fix bugs in PlaylistEntries 2022-11-12 00:02:12 +05:30
AudreyandGitHub d965856235 [extractor/Veoh] Add user extractor (#5242)
Authored by: tntmod54321
2022-11-11 23:28:54 +05:30
pukkandan 08270da5c3 [extractor/youtube] Fix ytuser: 2022-11-11 16:29:52 +05:30
744 changed files with 13328 additions and 5677 deletions
+6 -6
View File
@@ -7,7 +7,7 @@ body:
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
description: Fill all fields even if you think it is irrelevant for the issue description: Fill all fields even if you think it is irrelevant for the issue
options: options:
- label: I understand that I will be **blocked** if I remove or skip any mandatory\* field - label: I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field
required: true required: true
- type: checkboxes - type: checkboxes
id: checklist id: checklist
@@ -18,13 +18,13 @@ body:
options: options:
- label: I'm reporting a broken site - label: I'm reporting a broken site
required: true required: true
- label: I've verified that I'm running yt-dlp version **2022.11.11** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit) - label: I've verified that I'm running yt-dlp version **2023.02.17** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
required: true required: true
- label: I've checked that all provided URLs are playable in a browser with the same IP and same login details - label: I've checked that all provided URLs are playable in a browser with the same IP and same login details
required: true required: true
- label: I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command) - label: I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
required: true required: true
- label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates - label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
required: true required: true
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
required: true required: true
@@ -62,7 +62,7 @@ body:
[debug] Command-line config: ['-vU', 'test:youtube'] [debug] Command-line config: ['-vU', 'test:youtube']
[debug] Portable config "yt-dlp.conf": ['-i'] [debug] Portable config "yt-dlp.conf": ['-i']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8 [debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [9d339c4] (win32_exe) [debug] yt-dlp version 2023.02.17 [9d339c4] (win32_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0 [debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0
[debug] Checking exe version: ffmpeg -bsfs [debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs [debug] Checking exe version: ffprobe -bsfs
@@ -70,8 +70,8 @@ body:
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3 [debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {} [debug] Proxy map: {}
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest [debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11 Latest version: 2023.02.17, Current version: 2023.02.17
yt-dlp is up to date (2022.11.11) yt-dlp is up to date (2023.02.17)
<more lines> <more lines>
render: shell render: shell
validations: validations:
@@ -7,7 +7,7 @@ body:
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
description: Fill all fields even if you think it is irrelevant for the issue description: Fill all fields even if you think it is irrelevant for the issue
options: options:
- label: I understand that I will be **blocked** if I remove or skip any mandatory\* field - label: I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field
required: true required: true
- type: checkboxes - type: checkboxes
id: checklist id: checklist
@@ -18,13 +18,13 @@ body:
options: options:
- label: I'm reporting a new site support request - label: I'm reporting a new site support request
required: true required: true
- label: I've verified that I'm running yt-dlp version **2022.11.11** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit) - label: I've verified that I'm running yt-dlp version **2023.02.17** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
required: true required: true
- label: I've checked that all provided URLs are playable in a browser with the same IP and same login details - label: I've checked that all provided URLs are playable in a browser with the same IP and same login details
required: true required: true
- label: I've checked that none of provided URLs [violate any copyrights](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#is-the-website-primarily-used-for-piracy) or contain any [DRM](https://en.wikipedia.org/wiki/Digital_rights_management) to the best of my knowledge - label: I've checked that none of provided URLs [violate any copyrights](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#is-the-website-primarily-used-for-piracy) or contain any [DRM](https://en.wikipedia.org/wiki/Digital_rights_management) to the best of my knowledge
required: true required: true
- label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates - label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
required: true required: true
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
required: true required: true
@@ -74,7 +74,7 @@ body:
[debug] Command-line config: ['-vU', 'test:youtube'] [debug] Command-line config: ['-vU', 'test:youtube']
[debug] Portable config "yt-dlp.conf": ['-i'] [debug] Portable config "yt-dlp.conf": ['-i']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8 [debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [9d339c4] (win32_exe) [debug] yt-dlp version 2023.02.17 [9d339c4] (win32_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0 [debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0
[debug] Checking exe version: ffmpeg -bsfs [debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs [debug] Checking exe version: ffprobe -bsfs
@@ -82,8 +82,8 @@ body:
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3 [debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {} [debug] Proxy map: {}
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest [debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11 Latest version: 2023.02.17, Current version: 2023.02.17
yt-dlp is up to date (2022.11.11) yt-dlp is up to date (2023.02.17)
<more lines> <more lines>
render: shell render: shell
validations: validations:
@@ -7,7 +7,7 @@ body:
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
description: Fill all fields even if you think it is irrelevant for the issue description: Fill all fields even if you think it is irrelevant for the issue
options: options:
- label: I understand that I will be **blocked** if I remove or skip any mandatory\* field - label: I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field
required: true required: true
- type: checkboxes - type: checkboxes
id: checklist id: checklist
@@ -18,11 +18,11 @@ body:
options: options:
- label: I'm requesting a site-specific feature - label: I'm requesting a site-specific feature
required: true required: true
- label: I've verified that I'm running yt-dlp version **2022.11.11** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit) - label: I've verified that I'm running yt-dlp version **2023.02.17** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
required: true required: true
- label: I've checked that all provided URLs are playable in a browser with the same IP and same login details - label: I've checked that all provided URLs are playable in a browser with the same IP and same login details
required: true required: true
- label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates - label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
required: true required: true
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
required: true required: true
@@ -70,7 +70,7 @@ body:
[debug] Command-line config: ['-vU', 'test:youtube'] [debug] Command-line config: ['-vU', 'test:youtube']
[debug] Portable config "yt-dlp.conf": ['-i'] [debug] Portable config "yt-dlp.conf": ['-i']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8 [debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [9d339c4] (win32_exe) [debug] yt-dlp version 2023.02.17 [9d339c4] (win32_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0 [debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0
[debug] Checking exe version: ffmpeg -bsfs [debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs [debug] Checking exe version: ffprobe -bsfs
@@ -78,8 +78,8 @@ body:
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3 [debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {} [debug] Proxy map: {}
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest [debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11 Latest version: 2023.02.17, Current version: 2023.02.17
yt-dlp is up to date (2022.11.11) yt-dlp is up to date (2023.02.17)
<more lines> <more lines>
render: shell render: shell
validations: validations:
+6 -6
View File
@@ -7,7 +7,7 @@ body:
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
description: Fill all fields even if you think it is irrelevant for the issue description: Fill all fields even if you think it is irrelevant for the issue
options: options:
- label: I understand that I will be **blocked** if I remove or skip any mandatory\* field - label: I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field
required: true required: true
- type: checkboxes - type: checkboxes
id: checklist id: checklist
@@ -18,13 +18,13 @@ body:
options: options:
- label: I'm reporting a bug unrelated to a specific site - label: I'm reporting a bug unrelated to a specific site
required: true required: true
- label: I've verified that I'm running yt-dlp version **2022.11.11** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit) - label: I've verified that I'm running yt-dlp version **2023.02.17** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
required: true required: true
- label: I've checked that all provided URLs are playable in a browser with the same IP and same login details - label: I've checked that all provided URLs are playable in a browser with the same IP and same login details
required: true required: true
- label: I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command) - label: I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
required: true required: true
- label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates - label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
required: true required: true
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
required: true required: true
@@ -55,7 +55,7 @@ body:
[debug] Command-line config: ['-vU', 'test:youtube'] [debug] Command-line config: ['-vU', 'test:youtube']
[debug] Portable config "yt-dlp.conf": ['-i'] [debug] Portable config "yt-dlp.conf": ['-i']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8 [debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [9d339c4] (win32_exe) [debug] yt-dlp version 2023.02.17 [9d339c4] (win32_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0 [debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0
[debug] Checking exe version: ffmpeg -bsfs [debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs [debug] Checking exe version: ffprobe -bsfs
@@ -63,8 +63,8 @@ body:
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3 [debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {} [debug] Proxy map: {}
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest [debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11 Latest version: 2023.02.17, Current version: 2023.02.17
yt-dlp is up to date (2022.11.11) yt-dlp is up to date (2023.02.17)
<more lines> <more lines>
render: shell render: shell
validations: validations:
+6 -6
View File
@@ -7,7 +7,7 @@ body:
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
description: Fill all fields even if you think it is irrelevant for the issue description: Fill all fields even if you think it is irrelevant for the issue
options: options:
- label: I understand that I will be **blocked** if I remove or skip any mandatory\* field - label: I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field
required: true required: true
- type: checkboxes - type: checkboxes
id: checklist id: checklist
@@ -20,9 +20,9 @@ body:
required: true required: true
- label: I've looked through the [README](https://github.com/yt-dlp/yt-dlp#readme) - label: I've looked through the [README](https://github.com/yt-dlp/yt-dlp#readme)
required: true required: true
- label: I've verified that I'm running yt-dlp version **2022.11.11** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit) - label: I've verified that I'm running yt-dlp version **2023.02.17** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
required: true required: true
- label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates - label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
required: true required: true
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
required: true required: true
@@ -51,7 +51,7 @@ body:
[debug] Command-line config: ['-vU', 'test:youtube'] [debug] Command-line config: ['-vU', 'test:youtube']
[debug] Portable config "yt-dlp.conf": ['-i'] [debug] Portable config "yt-dlp.conf": ['-i']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8 [debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [9d339c4] (win32_exe) [debug] yt-dlp version 2023.02.17 [9d339c4] (win32_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0 [debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0
[debug] Checking exe version: ffmpeg -bsfs [debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs [debug] Checking exe version: ffprobe -bsfs
@@ -59,7 +59,7 @@ body:
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3 [debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {} [debug] Proxy map: {}
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest [debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11 Latest version: 2023.02.17, Current version: 2023.02.17
yt-dlp is up to date (2022.11.11) yt-dlp is up to date (2023.02.17)
<more lines> <more lines>
render: shell render: shell
+6 -6
View File
@@ -7,7 +7,7 @@ body:
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
description: Fill all fields even if you think it is irrelevant for the issue description: Fill all fields even if you think it is irrelevant for the issue
options: options:
- label: I understand that I will be **blocked** if I remove or skip any mandatory\* field - label: I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field
required: true required: true
- type: markdown - type: markdown
attributes: attributes:
@@ -26,9 +26,9 @@ body:
required: true required: true
- label: I've looked through the [README](https://github.com/yt-dlp/yt-dlp#readme) - label: I've looked through the [README](https://github.com/yt-dlp/yt-dlp#readme)
required: true required: true
- label: I've verified that I'm running yt-dlp version **2022.11.11** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit) - label: I've verified that I'm running yt-dlp version **2023.02.17** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
required: true required: true
- label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar questions **including closed ones**. DO NOT post duplicates - label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar questions **including closed ones**. DO NOT post duplicates
required: true required: true
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
required: true required: true
@@ -57,7 +57,7 @@ body:
[debug] Command-line config: ['-vU', 'test:youtube'] [debug] Command-line config: ['-vU', 'test:youtube']
[debug] Portable config "yt-dlp.conf": ['-i'] [debug] Portable config "yt-dlp.conf": ['-i']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8 [debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [9d339c4] (win32_exe) [debug] yt-dlp version 2023.02.17 [9d339c4] (win32_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0 [debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0
[debug] Checking exe version: ffmpeg -bsfs [debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs [debug] Checking exe version: ffprobe -bsfs
@@ -65,7 +65,7 @@ body:
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3 [debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {} [debug] Proxy map: {}
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest [debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11 Latest version: 2023.02.17, Current version: 2023.02.17
yt-dlp is up to date (2022.11.11) yt-dlp is up to date (2023.02.17)
<more lines> <more lines>
render: shell render: shell
@@ -18,7 +18,7 @@ body:
required: true required: true
- label: I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command) - label: I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
required: true required: true
- label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates - label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
required: true required: true
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
required: true required: true
@@ -18,7 +18,7 @@ body:
required: true required: true
- label: I've checked that none of provided URLs [violate any copyrights](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#is-the-website-primarily-used-for-piracy) or contain any [DRM](https://en.wikipedia.org/wiki/Digital_rights_management) to the best of my knowledge - label: I've checked that none of provided URLs [violate any copyrights](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#is-the-website-primarily-used-for-piracy) or contain any [DRM](https://en.wikipedia.org/wiki/Digital_rights_management) to the best of my knowledge
required: true required: true
- label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates - label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
required: true required: true
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
required: true required: true
@@ -16,7 +16,7 @@ body:
required: true required: true
- label: I've checked that all provided URLs are playable in a browser with the same IP and same login details - label: I've checked that all provided URLs are playable in a browser with the same IP and same login details
required: true required: true
- label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates - label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
required: true required: true
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
required: true required: true
+1 -1
View File
@@ -18,7 +18,7 @@ body:
required: true required: true
- label: I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command) - label: I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
required: true required: true
- label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates - label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
required: true required: true
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
required: true required: true
@@ -16,7 +16,7 @@ body:
required: true required: true
- label: I've verified that I'm running yt-dlp version **%(version)s** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit) - label: I've verified that I'm running yt-dlp version **%(version)s** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
required: true required: true
- label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates - label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
required: true required: true
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
required: true required: true
+1 -1
View File
@@ -22,7 +22,7 @@ body:
required: true required: true
- label: I've verified that I'm running yt-dlp version **%(version)s** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit) - label: I've verified that I'm running yt-dlp version **%(version)s** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
required: true required: true
- label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar questions **including closed ones**. DO NOT post duplicates - label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar questions **including closed ones**. DO NOT post duplicates
required: true required: true
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
required: true required: true
+3 -3
View File
@@ -2,8 +2,6 @@
### Description of your *pull request* and other information ### Description of your *pull request* and other information
</details>
<!-- <!--
Explanation of your *pull request* in arbitrary form goes here. Please **make sure the description explains the purpose and effect** of your *pull request* and is worded well enough to be understood. Provide as much **context and examples** as possible Explanation of your *pull request* in arbitrary form goes here. Please **make sure the description explains the purpose and effect** of your *pull request* and is worded well enough to be understood. Provide as much **context and examples** as possible
@@ -32,7 +30,7 @@ Fixes #
- [ ] [Searched](https://github.com/yt-dlp/yt-dlp/search?q=is%3Apr&type=Issues) the bugtracker for similar pull requests - [ ] [Searched](https://github.com/yt-dlp/yt-dlp/search?q=is%3Apr&type=Issues) the bugtracker for similar pull requests
- [ ] Checked the code with [flake8](https://pypi.python.org/pypi/flake8) and [ran relevant tests](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions) - [ ] Checked the code with [flake8](https://pypi.python.org/pypi/flake8) and [ran relevant tests](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions)
### In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under [Unlicense](http://unlicense.org/). Check one of the following options: ### In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under [Unlicense](http://unlicense.org/). Check all of the following options that apply:
- [ ] I am the original author of this code and I am willing to release it under [Unlicense](http://unlicense.org/) - [ ] I am the original author of this code and I am willing to release it under [Unlicense](http://unlicense.org/)
- [ ] I am not the original author of this code but it is in public domain or released under [Unlicense](http://unlicense.org/) (provide reliable evidence) - [ ] I am not the original author of this code but it is in public domain or released under [Unlicense](http://unlicense.org/) (provide reliable evidence)
@@ -41,3 +39,5 @@ Fixes #
- [ ] New extractor ([Piracy websites will not be accepted](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#is-the-website-primarily-used-for-piracy)) - [ ] New extractor ([Piracy websites will not be accepted](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#is-the-website-primarily-used-for-piracy))
- [ ] Core bug fix/improvement - [ ] Core bug fix/improvement
- [ ] New feature (It is strongly [recommended to open an issue first](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#adding-new-feature-or-making-overarching-changes)) - [ ] New feature (It is strongly [recommended to open an issue first](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#adding-new-feature-or-making-overarching-changes))
</details>
+2 -2
View File
@@ -255,7 +255,7 @@ jobs:
- name: Install Requirements - name: Install Requirements
run: | # Custom pyinstaller built with https://github.com/yt-dlp/pyinstaller-builds run: | # Custom pyinstaller built with https://github.com/yt-dlp/pyinstaller-builds
python -m pip install -U pip setuptools wheel py2exe python -m pip install -U pip setuptools wheel py2exe
pip install -U "https://yt-dlp.github.io/Pyinstaller-Builds/x86_64/pyinstaller-5.3-py3-none-any.whl" -r requirements.txt pip install -U "https://yt-dlp.github.io/Pyinstaller-Builds/x86_64/pyinstaller-5.8.0-py3-none-any.whl" -r requirements.txt
- name: Prepare - name: Prepare
run: | run: |
@@ -291,7 +291,7 @@ jobs:
- name: Install Requirements - name: Install Requirements
run: | run: |
python -m pip install -U pip setuptools wheel python -m pip install -U pip setuptools wheel
pip install -U "https://yt-dlp.github.io/Pyinstaller-Builds/i686/pyinstaller-5.3-py3-none-any.whl" -r requirements.txt pip install -U "https://yt-dlp.github.io/Pyinstaller-Builds/i686/pyinstaller-5.8.0-py3-none-any.whl" -r requirements.txt
- name: Prepare - name: Prepare
run: | run: |
+6 -5
View File
@@ -12,13 +12,13 @@ jobs:
fail-fast: false fail-fast: false
matrix: matrix:
os: [ubuntu-latest] os: [ubuntu-latest]
# CPython 3.9 is in quick-test # CPython 3.11 is in quick-test
python-version: ['3.7', '3.10', 3.11-dev, pypy-3.7, pypy-3.8] python-version: ['3.8', '3.9', '3.10', pypy-3.7, pypy-3.8]
run-tests-ext: [sh] run-tests-ext: [sh]
include: include:
# atleast one of each CPython/PyPy tests must be in windows # atleast one of each CPython/PyPy tests must be in windows
- os: windows-latest - os: windows-latest
python-version: '3.8' python-version: '3.7'
run-tests-ext: bat run-tests-ext: bat
- os: windows-latest - os: windows-latest
python-version: pypy-3.9 python-version: pypy-3.9
@@ -33,5 +33,6 @@ jobs:
run: pip install pytest run: pip install pytest
- name: Run tests - name: Run tests
continue-on-error: False continue-on-error: False
run: ./devscripts/run_tests.${{ matrix.run-tests-ext }} core run: |
# Linter is in quick-test python3 -m yt_dlp -v || true # Print debug head
./devscripts/run_tests.${{ matrix.run-tests-ext }} core
+6 -7
View File
@@ -10,24 +10,23 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v3 - uses: actions/checkout@v3
- name: Set up Python - name: Set up Python 3.11
uses: actions/setup-python@v4 uses: actions/setup-python@v4
with: with:
python-version: 3.9 python-version: '3.11'
- name: Install test requirements - name: Install test requirements
run: pip install pytest pycryptodomex run: pip install pytest pycryptodomex
- name: Run tests - name: Run tests
run: ./devscripts/run_tests.sh core run: |
python3 -m yt_dlp -v || true
./devscripts/run_tests.sh core
flake8: flake8:
name: Linter name: Linter
if: "!contains(github.event.head_commit.message, 'ci skip all')" if: "!contains(github.event.head_commit.message, 'ci skip all')"
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v3 - uses: actions/checkout@v3
- name: Set up Python - uses: actions/setup-python@v4
uses: actions/setup-python@v4
with:
python-version: 3.9
- name: Install flake8 - name: Install flake8
run: pip install flake8 run: pip install flake8
- name: Make lazy extractors - name: Make lazy extractors
+4 -6
View File
@@ -30,6 +30,7 @@ cookies
*.f4v *.f4v
*.flac *.flac
*.flv *.flv
*.gif
*.jpeg *.jpeg
*.jpg *.jpg
*.m4a *.m4a
@@ -71,6 +72,7 @@ dist/
zip/ zip/
tmp/ tmp/
venv/ venv/
.venv/
completions/ completions/
# Misc # Misc
@@ -119,9 +121,5 @@ yt-dlp.zip
*/extractor/lazy_extractors.py */extractor/lazy_extractors.py
# Plugins # Plugins
ytdlp_plugins/extractor/* ytdlp_plugins/
!ytdlp_plugins/extractor/__init__.py yt-dlp-plugins
!ytdlp_plugins/extractor/sample.py
ytdlp_plugins/postprocessor/*
!ytdlp_plugins/postprocessor/__init__.py
!ytdlp_plugins/postprocessor/sample.py
+19 -3
View File
@@ -351,8 +351,9 @@ Say you extracted a list of thumbnails into `thumbnail_data` and want to iterate
```python ```python
thumbnail_data = data.get('thumbnails') or [] thumbnail_data = data.get('thumbnails') or []
thumbnails = [{ thumbnails = [{
'url': item['url'] 'url': item['url'],
} for item in thumbnail_data] # correct 'height': item.get('h'),
} for item in thumbnail_data if item.get('url')] # correct
``` ```
and not like: and not like:
@@ -360,12 +361,27 @@ and not like:
```python ```python
thumbnail_data = data.get('thumbnails') thumbnail_data = data.get('thumbnails')
thumbnails = [{ thumbnails = [{
'url': item['url'] 'url': item['url'],
'height': item.get('h'),
} for item in thumbnail_data] # incorrect } for item in thumbnail_data] # incorrect
``` ```
In this case, `thumbnail_data` will be `None` if the field was not found and this will cause the loop `for item in thumbnail_data` to raise a fatal error. Using `or []` avoids this error and results in setting an empty list in `thumbnails` instead. In this case, `thumbnail_data` will be `None` if the field was not found and this will cause the loop `for item in thumbnail_data` to raise a fatal error. Using `or []` avoids this error and results in setting an empty list in `thumbnails` instead.
Alternately, this can be further simplified by using `traverse_obj`
```python
thumbnails = [{
'url': item['url'],
'height': item.get('h'),
} for item in traverse_obj(data, ('thumbnails', lambda _, v: v['url']))]
```
or, even better,
```python
thumbnails = traverse_obj(data, ('thumbnails', ..., {'url': 'url', 'height': 'h'}))
```
### Provide fallbacks ### Provide fallbacks
+50 -2
View File
@@ -3,6 +3,8 @@ shirt-dev (collaborator)
coletdjnz/colethedj (collaborator) coletdjnz/colethedj (collaborator)
Ashish0804 (collaborator) Ashish0804 (collaborator)
nao20010128nao/Lesmiscore (collaborator) nao20010128nao/Lesmiscore (collaborator)
bashonly (collaborator)
Grub4K (collaborator)
h-h-h-h h-h-h-h
pauldubois98 pauldubois98
nixxo nixxo
@@ -295,7 +297,6 @@ Mehavoid
winterbird-code winterbird-code
yashkc2025 yashkc2025
aldoridhoni aldoridhoni
bashonly
jacobtruman jacobtruman
masta79 masta79
palewire palewire
@@ -319,7 +320,6 @@ columndeeply
DoubleCouponDay DoubleCouponDay
Fabi019 Fabi019
GautamMKGarg GautamMKGarg
Grub4K
itachi-19 itachi-19
jeroenj jeroenj
josanabr josanabr
@@ -357,3 +357,51 @@ SG5
the-marenga the-marenga
tkgmomosheep tkgmomosheep
vitkhab vitkhab
glensc
synthpop123
tntmod54321
milkknife
Bnyro
CapacitorSet
stelcodes
skbeh
muddi900
digitall
chengzhicn
mexus
JChris246
redraskal
Spicadox
barsnick
docbender
KurtBestor
Chrissi2812
FrederikNS
gschizas
JC-Chung
mzhou
OndrejBakan
ab4cbef
aionescu
amra
ByteDream
carusocr
chexxor
felixonmars
FrankZ85
FriedrichRehren
gregsadetsky
LeoniePhiline
LowSuggestion912
Matumo
OIRNOIR
OMEGARAZER
oxamun
pmitchell86
qbnu
qulaz
rebane2001
road-master
rohieb
sdht0
seproDev
+404
View File
@@ -10,6 +10,410 @@
* Dispatch the workflow https://github.com/yt-dlp/yt-dlp/actions/workflows/build.yml on master * Dispatch the workflow https://github.com/yt-dlp/yt-dlp/actions/workflows/build.yml on master
--> -->
# 2023.02.17
* Merge youtube-dl: Upto [commit/2dd6c6e](https://github.com/ytdl-org/youtube-dl/commit/2dd6c6e)
* Fix `--concat-playlist`
* Imply `--no-progress` when `--print`
* Improve default subtitle language selection by [sdht0](https://github.com/sdht0)
* Make `title` completely non-fatal
* Sanitize formats before sorting by [pukkandan](https://github.com/pukkandan)
* Support module level `__bool__` and `property`
* [dependencies] Standardize `Cryptodome` imports
* [hls] Allow extractors to provide AES key by [Grub4K](https://github.com/Grub4K), [bashonly](https://github.com/bashonly)
* [ExtractAudio] Handle outtmpl without ext by [carusocr](https://github.com/carusocr)
* [extractor/common] Fix `_search_nuxt_data` by [LowSuggestion912](https://github.com/LowSuggestion912)
* [extractor/generic] Avoid catastrophic backtracking in KVS regex by [bashonly](https://github.com/bashonly)
* [jsinterp] Support `if` statements
* [plugins] Fix zip search paths
* [utils] `traverse_obj`: Various improvements by [Grub4K](https://github.com/Grub4K)
* [utils] `traverse_obj`: Fix more bugs
* [utils] `traverse_obj`: Fix several behavioral problems by [Grub4K](https://github.com/Grub4K)
* [utils] Don't use Content-length with encoding by [felixonmars](https://github.com/felixonmars)
* [utils] Fix `time_seconds` to use the provided TZ by [Grub4K](https://github.com/Grub4K), [Lesmiscore](https://github.com/Lesmiscore)
* [utils] Fix race condition in `make_dir` by [aionescu](https://github.com/aionescu)
* [utils] Use local kernel32 for file locking on Windows by [Grub4K](https://github.com/Grub4K)
* [compat_utils] Improve `passthrough_module`
* [compat_utils] Simplify `EnhancedModule`
* [build] Update pyinstaller
* [pyinst] Fix for pyinstaller 5.8
* [devscripts] Provide `pyinstaller` hooks
* [devscripts/pyinstaller] Analyze sub-modules of `Cryptodome`
* [cleanup] Misc fixes and cleanup
* [extractor/anchorfm] Add episode extractor by [HobbyistDev](https://github.com/HobbyistDev), [bashonly](https://github.com/bashonly)
* [extractor/boxcast] Add extractor by [HobbyistDev](https://github.com/HobbyistDev)
* [extractor/ebay] Add extractor by [JChris246](https://github.com/JChris246)
* [extractor/hypergryph] Add extractor by [HobbyistDev](https://github.com/HobbyistDev), [bashonly](https://github.com/bashonly)
* [extractor/NZOnScreen] Add extractor by [gregsadetsky](https://github.com/gregsadetsky), [pukkandan](https://github.com/pukkandan)
* [extractor/rozhlas] Add extractor RozhlasVltavaIE by [amra](https://github.com/amra)
* [extractor/tempo] Add IVXPlayer extractor by [HobbyistDev](https://github.com/HobbyistDev)
* [extractor/txxx] Add extractors by [chio0hai](https://github.com/chio0hai)
* [extractor/vocaroo] Add extractor by [SuperSonicHub1](https://github.com/SuperSonicHub1), [qbnu](https://github.com/qbnu)
* [extractor/wrestleuniverse] Add extractors by [Grub4K](https://github.com/Grub4K), [bashonly](https://github.com/bashonly)
* [extractor/yappy] Add extractor by [HobbyistDev](https://github.com/HobbyistDev)
* **[extractor/youtube] Fix `uploader_id` extraction** by [bashonly](https://github.com/bashonly)
* [extractor/youtube] Add hyperpipe instances by [Generator](https://github.com/Generator)
* [extractor/youtube] Handle `consent.youtube`
* [extractor/youtube] Support `/live/` URL
* [extractor/youtube] Update invidious and piped instances by [rohieb](https://github.com/rohieb)
* [extractor/91porn] Fix title and comment extraction by [pmitchell86](https://github.com/pmitchell86)
* [extractor/AbemaTV] Cache user token whenever appropriate by [Lesmiscore](https://github.com/Lesmiscore)
* [extractor/bfmtv] Support `rmc` prefix by [carusocr](https://github.com/carusocr)
* [extractor/biliintl] Add intro and ending chapters by [HobbyistDev](https://github.com/HobbyistDev)
* [extractor/clyp] Support `wav` by [qulaz](https://github.com/qulaz)
* [extractor/crunchyroll] Add intro chapter by [ByteDream](https://github.com/ByteDream)
* [extractor/crunchyroll] Better message for premium videos
* [extractor/crunchyroll] Fix incorrect premium-only error by [Grub4K](https://github.com/Grub4K)
* [extractor/DouyuTV] Use new API by [hatienl0i261299](https://github.com/hatienl0i261299)
* [extractor/embedly] Embedded links may be for other extractors
* [extractor/freesound] Workaround invalid URL in webpage by [rebane2001](https://github.com/rebane2001)
* [extractor/GoPlay] Use new API by [jeroenj](https://github.com/jeroenj)
* [extractor/Hidive] Fix subtitles and age-restriction by [chexxor](https://github.com/chexxor)
* [extractor/huya] Support HD streams by [felixonmars](https://github.com/felixonmars)
* [extractor/moviepilot] Fix extractor by [panatexxa](https://github.com/panatexxa)
* [extractor/nbc] Fix `NBC` and `NBCStations` extractors by [bashonly](https://github.com/bashonly)
* [extractor/nbc] Fix XML parsing by [bashonly](https://github.com/bashonly)
* [extractor/nebula] Remove broken cookie support by [hheimbuerger](https://github.com/hheimbuerger)
* [extractor/nfl] Add `NFLPlus` extractors by [bashonly](https://github.com/bashonly)
* [extractor/niconico] Add support for like history by [Matumo](https://github.com/Matumo), [pukkandan](https://github.com/pukkandan)
* [extractor/nitter] Update instance list by [OIRNOIR](https://github.com/OIRNOIR)
* [extractor/npo] Fix extractor and add HD support by [seproDev](https://github.com/seproDev)
* [extractor/odkmedia] Add `OnDemandChinaEpisodeIE` by [HobbyistDev](https://github.com/HobbyistDev), [pukkandan](https://github.com/pukkandan)
* [extractor/pornez] Handle relative URLs in iframe by [JChris246](https://github.com/JChris246)
* [extractor/radiko] Fix format sorting for Time Free by [road-master](https://github.com/road-master)
* [extractor/rcs] Fix extractors by [nixxo](https://github.com/nixxo), [pukkandan](https://github.com/pukkandan)
* [extractor/reddit] Support user posts by [OMEGARAZER](https://github.com/OMEGARAZER)
* [extractor/rumble] Fix format sorting by [pukkandan](https://github.com/pukkandan)
* [extractor/servus] Rewrite extractor by [Ashish0804](https://github.com/Ashish0804), [FrankZ85](https://github.com/FrankZ85), [StefanLobbenmeier](https://github.com/StefanLobbenmeier)
* [extractor/slideslive] Fix slides and chapters/duration by [bashonly](https://github.com/bashonly)
* [extractor/SportDeutschland] Fix extractor by [FriedrichRehren](https://github.com/FriedrichRehren)
* [extractor/Stripchat] Fix extractor by [JChris246](https://github.com/JChris246), [bashonly](https://github.com/bashonly)
* [extractor/tnaflix] Fix extractor by [bashonly](https://github.com/bashonly), [oxamun](https://github.com/oxamun)
* [extractor/tvp] Support `stream.tvp.pl` by [selfisekai](https://github.com/selfisekai)
* [extractor/twitter] Fix `--no-playlist` and add media `view_count` when using GraphQL by [Grub4K](https://github.com/Grub4K)
* [extractor/twitter] Fix graphql extraction on some tweets by [selfisekai](https://github.com/selfisekai)
* [extractor/vimeo] Fix `playerConfig` extraction by [LeoniePhiline](https://github.com/LeoniePhiline), [bashonly](https://github.com/bashonly)
* [extractor/viu] Add `ViuOTTIndonesiaIE` extractor by [HobbyistDev](https://github.com/HobbyistDev)
* [extractor/vk] Fix playlists for new API by [the-marenga](https://github.com/the-marenga)
* [extractor/vlive] Replace with `VLiveWebArchiveIE` by [seproDev](https://github.com/seproDev)
* [extractor/ximalaya] Update album `_VALID_URL` by [carusocr](https://github.com/carusocr)
* [extractor/zdf] Use android API endpoint for UHD downloads by [seproDev](https://github.com/seproDev)
* [extractor/drtv] Fix bug in [ab4cbef](https://github.com/yt-dlp/yt-dlp/commit/ab4cbef) by [bashonly](https://github.com/bashonly)
### 2023.02.17
#### Core changes
### Core changes
- [Bugfix for 39f32f1715c0dffb7626dda7307db6388bb7abaa](https://github.com/yt-dlp/yt-dlp/commit/9ebac35577e61c3d25fafc959655fa3ab04ca7ef) by [pukkandan](https://github.com/pukkandan)
- [Bugfix for 39f32f1715c0dffb7626dda7307db6388bb7abaa](https://github.com/yt-dlp/yt-dlp/commit/c154302c588c3d4362cec4fc5545e7e5d2bcf7a3) by [pukkandan](https://github.com/pukkandan)
- [Fix `--concat-playlist`](https://github.com/yt-dlp/yt-dlp/commit/59d7de0da545944c48a82fc2937b996d7cd8cc9c) by [pukkandan](https://github.com/pukkandan)
- [Imply `--no-progress` when `--print`](https://github.com/yt-dlp/yt-dlp/commit/5712943b764ba819ef479524c32700228603817a) by [pukkandan](https://github.com/pukkandan)
- [Improve default subtitle language selection](https://github.com/yt-dlp/yt-dlp/commit/376aa24b1541e2bfb23337c0ae9bafa5bb3787f1) ([#6240](https://github.com/yt-dlp/yt-dlp/issues/6240)) by [sdht0](https://github.com/sdht0)
- [Make `title` completely non-fatal](https://github.com/yt-dlp/yt-dlp/commit/7aefd19afed357c80743405ec2ace2148cba42e3) by [pukkandan](https://github.com/pukkandan)
- [Sanitize formats before sorting](https://github.com/yt-dlp/yt-dlp/commit/39f32f1715c0dffb7626dda7307db6388bb7abaa) by [pukkandan](https://github.com/pukkandan)
- [Support module level `__bool__` and `property`](https://github.com/yt-dlp/yt-dlp/commit/754c84e2e416cf6609dd0e4632b4985a08d34043) by [pukkandan](https://github.com/pukkandan)
- [Update to ytdl-commit-2dd6c6e](https://github.com/yt-dlp/yt-dlp/commit/48fde8ac4ccbaaea868f6378814dde395f649fbf) by [pukkandan](https://github.com/pukkandan)
- [extractor/douyutv]: [Use new API](https://github.com/yt-dlp/yt-dlp/commit/f14c2333481c63c24017a41ded7d8f36726504b7) ([#6074](https://github.com/yt-dlp/yt-dlp/issues/6074)) by [hatienl0i261299](https://github.com/hatienl0i261299)
- compat_utils
- [Improve `passthrough_module`](https://github.com/yt-dlp/yt-dlp/commit/88426d9446758c707fb511408f2d6f56de952db4) by [pukkandan](https://github.com/pukkandan)
- [Simplify `EnhancedModule`](https://github.com/yt-dlp/yt-dlp/commit/768a00178109508893488e53a0e720b117fbccf6) by [pukkandan](https://github.com/pukkandan)
- dependencies
- [Standardize `Cryptodome` imports](https://github.com/yt-dlp/yt-dlp/commit/f6a765ceb59c55aea06921880c1c87d1ff36e5de) by [pukkandan](https://github.com/pukkandan)
- jsinterp
- [Support `if` statements](https://github.com/yt-dlp/yt-dlp/commit/8b008d62544b82e24a0ba36c30e8e51855d93419) by [pukkandan](https://github.com/pukkandan)
- plugins
- [Fix zip search paths](https://github.com/yt-dlp/yt-dlp/commit/88d8928bf7630801865cf8728ae5c77234324b7b) by [pukkandan](https://github.com/pukkandan)
- utils
- [Don't use Content-length with encoding](https://github.com/yt-dlp/yt-dlp/commit/65e5c021e7c5f23ecbc6a982b72a02ac6cd6900d) ([#6176](https://github.com/yt-dlp/yt-dlp/issues/6176)) by [felixonmars](https://github.com/felixonmars)
- [Fix `time_seconds` to use the provided TZ](https://github.com/yt-dlp/yt-dlp/commit/83c4970e52839ce8761ec61bd19d549aed7d7920) ([#6118](https://github.com/yt-dlp/yt-dlp/issues/6118)) by [Grub4K](https://github.com/Grub4K), [Lesmiscore](https://github.com/Lesmiscore)
- [Fix race condition in `make_dir`](https://github.com/yt-dlp/yt-dlp/commit/b25d6cb96337d479bdcb41768356da414c3aa835) ([#6089](https://github.com/yt-dlp/yt-dlp/issues/6089)) by [aionescu](https://github.com/aionescu)
- [Use local kernel32 for file locking on Windows](https://github.com/yt-dlp/yt-dlp/commit/37e325b92ff9d784715ac0e5d1f7d96bf5f45ad9) by [Grub4K](https://github.com/Grub4K)
- traverse_obj
- [Fix more bugs](https://github.com/yt-dlp/yt-dlp/commit/6839ae1f6dde4c0442619e351b3f0442312ab4f9) by [pukkandan](https://github.com/pukkandan)
- [Fix several behavioral problems](https://github.com/yt-dlp/yt-dlp/commit/b1bde57bef878478e3503ab07190fd207914ade9) by [Grub4K](https://github.com/Grub4K)
- [Various improvements](https://github.com/yt-dlp/yt-dlp/commit/776995bc109c5cd1aa56b684fada2ce718a386ec) by [Grub4K](https://github.com/Grub4K)
### Extractor changes
- [Fix `_search_nuxt_data`](https://github.com/yt-dlp/yt-dlp/commit/b23167e7542c177f32b22b29857b637dc4aede69) ([#6062](https://github.com/yt-dlp/yt-dlp/issues/6062)) by [LowSuggestion912](https://github.com/LowSuggestion912)
- 91porn
- [Fix title and comment extraction](https://github.com/yt-dlp/yt-dlp/commit/c085cc2def9862ac8a7619ce8ea5dcc177325719) ([#5932](https://github.com/yt-dlp/yt-dlp/issues/5932)) by [pmitchell86](https://github.com/pmitchell86)
- abematv
- [Cache user token whenever appropriate](https://github.com/yt-dlp/yt-dlp/commit/a4f16832213d9e29beecf685d6cd09a2f0b48c87) ([#6216](https://github.com/yt-dlp/yt-dlp/issues/6216)) by [Lesmiscore](https://github.com/Lesmiscore)
- anchorfm
- [Add episode extractor](https://github.com/yt-dlp/yt-dlp/commit/a4ad59ff2ded208bf33f6fe07299a3449eadccdc) ([#6092](https://github.com/yt-dlp/yt-dlp/issues/6092)) by [bashonly](https://github.com/bashonly), [HobbyistDev](https://github.com/HobbyistDev)
- bfmtv
- [Support `rmc` prefix](https://github.com/yt-dlp/yt-dlp/commit/20266508dd6247dd3cf0e97b9b9f14c3afc046db) ([#6025](https://github.com/yt-dlp/yt-dlp/issues/6025)) by [carusocr](https://github.com/carusocr)
- biliintl
- [Add intro and ending chapters](https://github.com/yt-dlp/yt-dlp/commit/0ba87dd279d3565ed93c559cf7880ad61eb83af8) ([#6018](https://github.com/yt-dlp/yt-dlp/issues/6018)) by [HobbyistDev](https://github.com/HobbyistDev)
- boxcast
- [Add extractor](https://github.com/yt-dlp/yt-dlp/commit/9acca71237f42a4775008e51fe26e42f0a39c552) ([#5983](https://github.com/yt-dlp/yt-dlp/issues/5983)) by [HobbyistDev](https://github.com/HobbyistDev)
- clyp
- [Support `wav`](https://github.com/yt-dlp/yt-dlp/commit/cc13293c2819b5461be211a9729fd02bb1e2f476) ([#6102](https://github.com/yt-dlp/yt-dlp/issues/6102)) by [qulaz](https://github.com/qulaz)
- crunchyroll
- [Add intro chapter](https://github.com/yt-dlp/yt-dlp/commit/93abb7406b95793f6872d12979b91d5f336b4f43) ([#6023](https://github.com/yt-dlp/yt-dlp/issues/6023)) by [ByteDream](https://github.com/ByteDream)
- [Better message for premium videos](https://github.com/yt-dlp/yt-dlp/commit/44699d10dc8de9c6a338f4a8e5c63506ec4d2118) by [pukkandan](https://github.com/pukkandan)
- [Fix incorrect premium-only error](https://github.com/yt-dlp/yt-dlp/commit/c9d14bd22ab31e2a41f9f8061843668a06db583b) by [Grub4K](https://github.com/Grub4K)
- drtv
- [Fix bug in ab4cbef](https://github.com/yt-dlp/yt-dlp/commit/7481998b169b2a52049fc33bff82034d6563ead4) ([#6034](https://github.com/yt-dlp/yt-dlp/issues/6034)) by [bashonly](https://github.com/bashonly)
- ebay
- [Add extractor](https://github.com/yt-dlp/yt-dlp/commit/da880559a6ecbbf374cc9f3378e696b55b9599af) ([#6170](https://github.com/yt-dlp/yt-dlp/issues/6170)) by [JChris246](https://github.com/JChris246)
- embedly
- [Embedded links may be for other extractors](https://github.com/yt-dlp/yt-dlp/commit/87ebab0615b1bf9b14b478b055e7059d630b4833) by [pukkandan](https://github.com/pukkandan)
- freesound
- [Workaround invalid URL in webpage](https://github.com/yt-dlp/yt-dlp/commit/9cfdbcbf3f17be51f5b6bb9bb6d880b2f3d67362) ([#6147](https://github.com/yt-dlp/yt-dlp/issues/6147)) by [rebane2001](https://github.com/rebane2001)
- generic
- [Avoid catastrophic backtracking in KVS regex](https://github.com/yt-dlp/yt-dlp/commit/8aa0bd5d10627ece3c1815c01d02fb8bf22847a7) by [bashonly](https://github.com/bashonly)
- goplay
- [Use new API](https://github.com/yt-dlp/yt-dlp/commit/d27bde98832e3b7ffb39f3cf6346011b97bb3bc3) ([#6151](https://github.com/yt-dlp/yt-dlp/issues/6151)) by [jeroenj](https://github.com/jeroenj)
- hidive
- [Fix subtitles and age-restriction](https://github.com/yt-dlp/yt-dlp/commit/7708df8da05c94270b43e0630e4e20f6d2d62c55) ([#5828](https://github.com/yt-dlp/yt-dlp/issues/5828)) by [chexxor](https://github.com/chexxor)
- huya
- [Support HD streams](https://github.com/yt-dlp/yt-dlp/commit/fbbb5508ea98ed8709847f5ecced7d70ff05e0ee) ([#6172](https://github.com/yt-dlp/yt-dlp/issues/6172)) by [felixonmars](https://github.com/felixonmars)
- hypergryph
- [Add extractor](https://github.com/yt-dlp/yt-dlp/commit/31c279a2a2c2ef402a9e6dad9992b310d16439a6) ([#6094](https://github.com/yt-dlp/yt-dlp/issues/6094)) by [bashonly](https://github.com/bashonly), [HobbyistDev](https://github.com/HobbyistDev)
- moviepilot
- [Fix extractor](https://github.com/yt-dlp/yt-dlp/commit/c62e64cf0122e52fa2175dd1b004ca6b8e1d82af) ([#5954](https://github.com/yt-dlp/yt-dlp/issues/5954)) by [panatexxa](https://github.com/panatexxa)
- nbc
- [Fix XML parsing](https://github.com/yt-dlp/yt-dlp/commit/176a068cde4f2d9dfa0336168caead0b1edcb8ac) by [bashonly](https://github.com/bashonly)
- [Fix `NBC` and `NBCStations` extractors](https://github.com/yt-dlp/yt-dlp/commit/cb73b8460c3ce6d37ab651a4e44bb23b10056154) ([#6033](https://github.com/yt-dlp/yt-dlp/issues/6033)) by [bashonly](https://github.com/bashonly)
- nebula
- [Remove broken cookie support](https://github.com/yt-dlp/yt-dlp/commit/d50ea3ce5abc3b0defc0e5d1e22b22ce9b01b07b) ([#5979](https://github.com/yt-dlp/yt-dlp/issues/5979)) by [hheimbuerger](https://github.com/hheimbuerger)
- nfl
- [Add `NFLPlus` extractors](https://github.com/yt-dlp/yt-dlp/commit/8b37c58f8b5494504acdb5ebe3f8bbd26230f725) ([#6222](https://github.com/yt-dlp/yt-dlp/issues/6222)) by [bashonly](https://github.com/bashonly)
- niconico
- [Add support for like history](https://github.com/yt-dlp/yt-dlp/commit/3b161265add30613bde2e46fca214fe94d09e651) ([#5705](https://github.com/yt-dlp/yt-dlp/issues/5705)) by [Matumo](https://github.com/Matumo), [pukkandan](https://github.com/pukkandan)
- nitter
- [Update instance list](https://github.com/yt-dlp/yt-dlp/commit/a9189510baadf0dccd2d4d363bc6f3a441128bb0) ([#6236](https://github.com/yt-dlp/yt-dlp/issues/6236)) by [OIRNOIR](https://github.com/OIRNOIR)
- npo
- [Fix extractor and add HD support](https://github.com/yt-dlp/yt-dlp/commit/cc2389c8ac72a514d4e002a0f6ca5a7d65c7eff0) ([#6155](https://github.com/yt-dlp/yt-dlp/issues/6155)) by [seproDev](https://github.com/seproDev)
- nzonscreen
- [Add extractor](https://github.com/yt-dlp/yt-dlp/commit/d3bb187f01e1e30db05e639fc23a2e1935d777fe) ([#6208](https://github.com/yt-dlp/yt-dlp/issues/6208)) by [gregsadetsky](https://github.com/gregsadetsky), [pukkandan](https://github.com/pukkandan)
- odkmedia
- [Add `OnDemandChinaEpisodeIE`](https://github.com/yt-dlp/yt-dlp/commit/10fd9e6ee833c88edf6c633f864f42843a708d32) ([#6116](https://github.com/yt-dlp/yt-dlp/issues/6116)) by [HobbyistDev](https://github.com/HobbyistDev), [pukkandan](https://github.com/pukkandan)
- pornez
- [Handle relative URLs in iframe](https://github.com/yt-dlp/yt-dlp/commit/f7efe6dc958eb0689cb9534ff0b4e592040be8df) ([#6171](https://github.com/yt-dlp/yt-dlp/issues/6171)) by [JChris246](https://github.com/JChris246)
- radiko
- [Fix format sorting for Time Free](https://github.com/yt-dlp/yt-dlp/commit/203a06f8554df6db07d8f20f465ecbfe8a14e591) ([#6159](https://github.com/yt-dlp/yt-dlp/issues/6159)) by [road-master](https://github.com/road-master)
- rcs
- [Fix extractors](https://github.com/yt-dlp/yt-dlp/commit/c6b657867ad68af6b930ed0aa11ec5d93ee187b7) ([#5700](https://github.com/yt-dlp/yt-dlp/issues/5700)) by [nixxo](https://github.com/nixxo), [pukkandan](https://github.com/pukkandan)
- reddit
- [Support user posts](https://github.com/yt-dlp/yt-dlp/commit/c77df98b1a477a020a57141464d10c0f4d0fdbc9) ([#6173](https://github.com/yt-dlp/yt-dlp/issues/6173)) by [OMEGARAZER](https://github.com/OMEGARAZER)
- rozhlas
- [Add extractor RozhlasVltavaIE](https://github.com/yt-dlp/yt-dlp/commit/355d781bed497cbcb254bf2a2737b83fa51c84ea) ([#5951](https://github.com/yt-dlp/yt-dlp/issues/5951)) by [amra](https://github.com/amra)
- rumble
- [Fix format sorting](https://github.com/yt-dlp/yt-dlp/commit/acacb57c7e173b93c6e0f0c43e61b9b2912719d8) by [pukkandan](https://github.com/pukkandan)
- servus
- [Rewrite extractor](https://github.com/yt-dlp/yt-dlp/commit/f40e32fb1ac67be5bdbc8e32a3c235abfc4be260) ([#6036](https://github.com/yt-dlp/yt-dlp/issues/6036)) by [Ashish0804](https://github.com/Ashish0804), [FrankZ85](https://github.com/FrankZ85), [StefanLobbenmeier](https://github.com/StefanLobbenmeier)
- slideslive
- [Fix slides and chapters/duration](https://github.com/yt-dlp/yt-dlp/commit/5ab3534d44231f7711398bc3cfc520e2efd09f50) ([#6024](https://github.com/yt-dlp/yt-dlp/issues/6024)) by [bashonly](https://github.com/bashonly)
- sportdeutschland
- [Fix extractor](https://github.com/yt-dlp/yt-dlp/commit/5e1a54f63e393c218a40949012ff0de0ce63cb15) ([#6041](https://github.com/yt-dlp/yt-dlp/issues/6041)) by [FriedrichRehren](https://github.com/FriedrichRehren)
- stripchat
- [Fix extractor](https://github.com/yt-dlp/yt-dlp/commit/7d5f919bad07017f4b39b55725491b1e9717d47a) ([#5985](https://github.com/yt-dlp/yt-dlp/issues/5985)) by [bashonly](https://github.com/bashonly), [JChris246](https://github.com/JChris246)
- tempo
- [Add IVXPlayer extractor](https://github.com/yt-dlp/yt-dlp/commit/30031be974d210f451100339699ef03b0ddb5f10) ([#5837](https://github.com/yt-dlp/yt-dlp/issues/5837)) by [HobbyistDev](https://github.com/HobbyistDev)
- tnaflix
- [Fix extractor](https://github.com/yt-dlp/yt-dlp/commit/989f47b6315541989bb507f26b431d9586430995) ([#6086](https://github.com/yt-dlp/yt-dlp/issues/6086)) by [bashonly](https://github.com/bashonly), [oxamun](https://github.com/oxamun)
- tvp
- [Support `stream.tvp.pl`](https://github.com/yt-dlp/yt-dlp/commit/a31d0fa6c315b1145d682361149003d98f1e3782) ([#6139](https://github.com/yt-dlp/yt-dlp/issues/6139)) by [selfisekai](https://github.com/selfisekai)
- twitter
- [Fix `--no-playlist` and add media `view_count` when using GraphQL](https://github.com/yt-dlp/yt-dlp/commit/b6795fd310f1dd61dddc9fd08e52fe485bdc8a3e) ([#6211](https://github.com/yt-dlp/yt-dlp/issues/6211)) by [Grub4K](https://github.com/Grub4K)
- [Fix graphql extraction on some tweets](https://github.com/yt-dlp/yt-dlp/commit/7543c9c99bcb116b085fdb1f41b84a0ead04c05d) ([#6075](https://github.com/yt-dlp/yt-dlp/issues/6075)) by [selfisekai](https://github.com/selfisekai)
- txxx
- [Add extractors](https://github.com/yt-dlp/yt-dlp/commit/389896df85ed14eaf74f72531da6c4491d6b73b0) ([#5240](https://github.com/yt-dlp/yt-dlp/issues/5240)) by [chio0hai](https://github.com/chio0hai)
- vimeo
- [Fix `playerConfig` extraction](https://github.com/yt-dlp/yt-dlp/commit/c0cd13fb1c71b842c3d272d0273c03542b467766) ([#6203](https://github.com/yt-dlp/yt-dlp/issues/6203)) by [bashonly](https://github.com/bashonly), [LeoniePhiline](https://github.com/LeoniePhiline)
- viu
- [Add `ViuOTTIndonesiaIE` extractor](https://github.com/yt-dlp/yt-dlp/commit/72671a212d7c939329cb5d34335fa089dd3acbd3) ([#6099](https://github.com/yt-dlp/yt-dlp/issues/6099)) by [HobbyistDev](https://github.com/HobbyistDev)
- vk
- [Fix playlists for new API](https://github.com/yt-dlp/yt-dlp/commit/a9c685453f7019bee94170f936619c6db76c964e) ([#6122](https://github.com/yt-dlp/yt-dlp/issues/6122)) by [the-marenga](https://github.com/the-marenga)
- vlive
- [Replace with `VLiveWebArchiveIE`](https://github.com/yt-dlp/yt-dlp/commit/b3eaab7ca2e118d4db73dcb44afd9c8717db8b67) ([#6196](https://github.com/yt-dlp/yt-dlp/issues/6196)) by [seproDev](https://github.com/seproDev)
- vocaroo
- [Add extractor](https://github.com/yt-dlp/yt-dlp/commit/e4a8b1769e19755acba6d8f212208359905a3159) ([#6117](https://github.com/yt-dlp/yt-dlp/issues/6117)) by [qbnu](https://github.com/qbnu), [SuperSonicHub1](https://github.com/SuperSonicHub1)
- wrestleuniverse
- [Add extractors](https://github.com/yt-dlp/yt-dlp/commit/e61acb40b2cb6ef45508d72235026d458c9d5dff) ([#6158](https://github.com/yt-dlp/yt-dlp/issues/6158)) by [bashonly](https://github.com/bashonly), [Grub4K](https://github.com/Grub4K)
- ximalaya
- [Update album `_VALID_URL`](https://github.com/yt-dlp/yt-dlp/commit/417cdaae08fc447c9d15c53a88e2e9a027cdbf0a) ([#6110](https://github.com/yt-dlp/yt-dlp/issues/6110)) by [carusocr](https://github.com/carusocr)
- yappy
- [Add extractor](https://github.com/yt-dlp/yt-dlp/commit/361630015535026712bdb67f804a15b65ff9ee7e) ([#6111](https://github.com/yt-dlp/yt-dlp/issues/6111)) by [HobbyistDev](https://github.com/HobbyistDev)
- youtube
- [Add hyperpipe instances](https://github.com/yt-dlp/yt-dlp/commit/78a78fa74dbc888d20f1b65e1382bf99131597d5) ([#6020](https://github.com/yt-dlp/yt-dlp/issues/6020)) by [Generator](https://github.com/Generator)
- [Fix `uploader_id` extraction](https://github.com/yt-dlp/yt-dlp/commit/149eb0bbf34fa8fdf8d1e2aa28e17479d099e26b) by [bashonly](https://github.com/bashonly)
- [Handle `consent.youtube`](https://github.com/yt-dlp/yt-dlp/commit/b032ff0f032512bd6fc70c9c1994d906eacc06cb) by [pukkandan](https://github.com/pukkandan)
- [Support `/live/` URL](https://github.com/yt-dlp/yt-dlp/commit/dad2210c0cb9cf03702a9511817ee5ec646d7bc8) by [pukkandan](https://github.com/pukkandan)
- [Update invidious and piped instances](https://github.com/yt-dlp/yt-dlp/commit/05799a48c7dec12b34c8bf951c8d2eceedda59f8) ([#6030](https://github.com/yt-dlp/yt-dlp/issues/6030)) by [rohieb](https://github.com/rohieb)
- [`uploader_id` includes `@` with handle](https://github.com/yt-dlp/yt-dlp/commit/c61cf091a54d3aa3c611722035ccde5ecfe981bb) by [bashonly](https://github.com/bashonly)
- zdf
- [Use android API endpoint for UHD downloads](https://github.com/yt-dlp/yt-dlp/commit/0fe87a8730638490415d630f48e61d264d89c358) ([#6150](https://github.com/yt-dlp/yt-dlp/issues/6150)) by [seproDev](https://github.com/seproDev)
### Downloader changes
- hls
- [Allow extractors to provide AES key](https://github.com/yt-dlp/yt-dlp/commit/7e68567e508168b345266c0c19812ad50a829eaa) ([#6158](https://github.com/yt-dlp/yt-dlp/issues/6158)) by [bashonly](https://github.com/bashonly), [Grub4K](https://github.com/Grub4K)
### Postprocessor changes
- extractaudio
- [Handle outtmpl without ext](https://github.com/yt-dlp/yt-dlp/commit/f737fb16d8234408c85bc189ccc926fea000515b) ([#6005](https://github.com/yt-dlp/yt-dlp/issues/6005)) by [carusocr](https://github.com/carusocr)
- pyinst
- [Fix for pyinstaller 5.8](https://github.com/yt-dlp/yt-dlp/commit/2e269bd998c61efaf7500907d114a56e5e83e65e) by [pukkandan](https://github.com/pukkandan)
### Misc. changes
- build
- [Update pyinstaller](https://github.com/yt-dlp/yt-dlp/commit/365b9006051ac7d735c20bb63c4907b758233048) by [pukkandan](https://github.com/pukkandan)
- cleanup
- Miscellaneous: [76c9c52](https://github.com/yt-dlp/yt-dlp/commit/76c9c523071150053df7b56956646b680b6a6e05) by [pukkandan](https://github.com/pukkandan)
- devscripts
- [Provide pyinstaller hooks](https://github.com/yt-dlp/yt-dlp/commit/acb1042a9ffa8769fe691beac1011d6da1fcf321) by [pukkandan](https://github.com/pukkandan)
- pyinstaller
- [Analyze sub-modules of `Cryptodome`](https://github.com/yt-dlp/yt-dlp/commit/b85faf6ffb700058e774e99c04304a7a9257cdd0) by [pukkandan](https://github.com/pukkandan)
### 2023.01.06
* Fix config locations by [Grub4k](https://github.com/Grub4k), [coletdjnz](https://github.com/coletdjnz), [pukkandan](https://github.com/pukkandan)
* [downloader/aria2c] Disable native progress
* [utils] `mimetype2ext`: `weba` is not standard
* [utils] `windows_enable_vt_mode`: Better error handling
* [build] Add minimal `pyproject.toml`
* [update] Fix updater file removal on windows by [Grub4K](https://github.com/Grub4K)
* [cleanup] Misc fixes and cleanup
* [extractor/aitube] Add extractor by [HobbyistDev](https://github.com/HobbyistDev)
* [extractor/drtv] Add series extractors by [FrederikNS](https://github.com/FrederikNS)
* [extractor/volejtv] Add extractor by [HobbyistDev](https://github.com/HobbyistDev)
* [extractor/xanimu] Add extractor by [JChris246](https://github.com/JChris246)
* [extractor/youtube] Retry manifest refresh for live-from-start by [mzhou](https://github.com/mzhou)
* [extractor/biliintl] Add `/media` to `VALID_URL` by [HobbyistDev](https://github.com/HobbyistDev)
* [extractor/biliIntl] Add fallback to `video_data` by [HobbyistDev](https://github.com/HobbyistDev)
* [extractor/crunchyroll:show] Add `language` to entries by [Chrissi2812](https://github.com/Chrissi2812)
* [extractor/joj] Fix extractor by [OndrejBakan](https://github.com/OndrejBakan), [pukkandan](https://github.com/pukkandan)
* [extractor/nbc] Update graphql query by [jacobtruman](https://github.com/jacobtruman)
* [extractor/reddit] Add subreddit as `channel_id` by [gschizas](https://github.com/gschizas)
* [extractor/tiktok] Add `TikTokLive` extractor by [JC-Chung](https://github.com/JC-Chung)
### 2023.01.02
* **Improve plugin architecture** by [Grub4K](https://github.com/Grub4K), [coletdjnz](https://github.com/coletdjnz), [flashdagger](https://github.com/flashdagger), [pukkandan](https://github.com/pukkandan)
* Plugins can be loaded in any distribution of yt-dlp (binary, pip, source, etc.) and can be distributed and installed as packages. See [the readme](https://github.com/yt-dlp/yt-dlp/tree/05997b6e98e638d97d409c65bb5eb86da68f3b64#plugins) for more information
* Add `--compat-options 2021,2022`
* This allows devs to change defaults and make other potentially breaking changes more easily. If you need everything to work exactly as-is, put Use `--compat 2022` in your config to guard against future compat changes.
* [downloader/aria2c] Native progress for aria2c via RPC by [Lesmiscore](https://github.com/Lesmiscore), [pukkandan](https://github.com/pukkandan)
* Merge youtube-dl: Upto [commit/195f22f](https://github.com/ytdl-org/youtube-dl/commit/195f22f6) by [Grub4k](https://github.com/Grub4k), [pukkandan](https://github.com/pukkandan)
* Add pre-processor stage `video`
* Let `--parse/replace-in-metadata` run at any post-processing stage
* Add `--enable-file-urls` by [coletdjnz](https://github.com/coletdjnz)
* Add new field `aspect_ratio`
* Add `ac4` to known codecs
* Add `weba` to known extensions
* [FFmpegVideoConvertor] Add `gif` to `--recode-video`
* Add message when there are no subtitles/thumbnails
* Deprioritize HEVC-over-FLV formats by [Lesmiscore](https://github.com/Lesmiscore)
* Make early reject of `--match-filter` stricter
* Fix `--cookies-from-browser` CLI parsing
* Fix `original_url` in playlists
* Fix bug in writing playlist info-json
* Fix bugs in `PlaylistEntries`
* [downloader/ffmpeg] Fix headers for video+audio formats by [Grub4K](https://github.com/Grub4K), [bashonly](https://github.com/bashonly)
* [extractor] Add a way to distinguish IEs that returns only videos
* [extractor] Implement universal format sorting and deprecate `_sort_formats`
* [extractor] Let `_extract_format` functions obey `--ignore-no-formats`
* [extractor/generic] Add `fragment_query` extractor arg for DASH and HLS by [bashonly](https://github.com/bashonly), [pukkandan](https://github.com/pukkandan)
* [extractor/generic] Decode unicode-escaped embed URLs by [bashonly](https://github.com/bashonly)
* [extractor/generic] Don't report redirect to https
* [extractor/generic] Fix JSON LD manifest extraction by [bashonly](https://github.com/bashonly), [pukkandan](https://github.com/pukkandan)
* [extractor/generic] Use `Accept-Encoding: identity` for initial request by [coletdjnz](https://github.com/coletdjnz)
* [FormatSort] Add `mov` to `vext`
* [jsinterp] Escape regex that looks like nested set
* [webvtt] Handle premature EOF by [flashdagger](https://github.com/flashdagger)
* [utils] `classproperty`: Add cache support
* [utils] `get_exe_version`: Detect broken executables by [dirkf](https://github.com/dirkf), [pukkandan](https://github.com/pukkandan)
* [utils] `js_to_json`: Fix bug in [f55523c](https://github.com/yt-dlp/yt-dlp/commit/f55523c) by [ChillingPepper](https://github.com/ChillingPepper), [pukkandan](https://github.com/pukkandan)
* [utils] Make `ExtractorError` mutable
* [utils] Move `FileDownloader.parse_bytes` into utils
* [utils] Move format sorting code into `utils`
* [utils] `windows_enable_vt_mode`: Proper implementation by [Grub4K](https://github.com/Grub4K)
* [update] Workaround [#5632](https://github.com/yt-dlp/yt-dlp/issues/5632)
* [docs] Improvements
* [cleanup] Misc fixes and cleanup
* [cleanup] Use `random.choices` by [freezboltz](https://github.com/freezboltz)
* [extractor/airtv] Add extractor by [HobbyistDev](https://github.com/HobbyistDev)
* [extractor/amazonminitv] Add extractors by [GautamMKGarg](https://github.com/GautamMKGarg), [nyuszika7h](https://github.com/nyuszika7h)
* [extractor/beatbump] Add extractors by [Bobscorn](https://github.com/Bobscorn), [pukkandan](https://github.com/pukkandan)
* [extractor/europarl] Add EuroParlWebstream extractor by [HobbyistDev](https://github.com/HobbyistDev)
* [extractor/kanal2] Add extractor by [bashonly](https://github.com/bashonly), [glensc](https://github.com/glensc), [pukkandan](https://github.com/pukkandan)
* [extractor/kankanews] Add extractor by [synthpop123](https://github.com/synthpop123)
* [extractor/kick] Add extractor by [bashonly](https://github.com/bashonly)
* [extractor/mediastream] Add extractor by [HobbyistDev](https://github.com/HobbyistDev), [elyse0](https://github.com/elyse0)
* [extractor/noice] Add NoicePodcast extractor by [HobbyistDev](https://github.com/HobbyistDev)
* [extractor/oneplace] Add OnePlacePodcast extractor by [HobbyistDev](https://github.com/HobbyistDev)
* [extractor/rumble] Add RumbleIE extractor by [flashdagger](https://github.com/flashdagger)
* [extractor/screencastify] Add extractor by [bashonly](https://github.com/bashonly)
* [extractor/trtcocuk] Add extractor by [HobbyistDev](https://github.com/HobbyistDev)
* [extractor/Veoh] Add user extractor by [tntmod54321](https://github.com/tntmod54321)
* [extractor/videoken] Add extractors by [bashonly](https://github.com/bashonly)
* [extractor/webcamerapl] Add extractor by [milkknife](https://github.com/milkknife)
* [extractor/amazon] Add `AmazonReviews` extractor by [bashonly](https://github.com/bashonly)
* [extractor/netverse] Add `NetverseSearch` extractor by [HobbyistDev](https://github.com/HobbyistDev)
* [extractor/vimeo] Add `VimeoProIE` by [bashonly](https://github.com/bashonly), [pukkandan](https://github.com/pukkandan)
* [extractor/xiami] Remove extractors by [synthpop123](https://github.com/synthpop123)
* [extractor/youtube] Add `piped.video` by [Bnyro](https://github.com/Bnyro)
* [extractor/youtube] Consider language in format de-duplication
* [extractor/youtube] Extract DRC formats
* [extractor/youtube] Fix `ytuser:`
* [extractor/youtube] Fix bug in handling of music URLs
* [extractor/youtube] Subtitles cannot be translated to `und`
* [extractor/youtube:tab] Extract metadata from channel items by [coletdjnz](https://github.com/coletdjnz)
* [extractor/ARD] Add vtt subtitles by [CapacitorSet](https://github.com/CapacitorSet)
* [extractor/ArteTV] Extract chapters by [bashonly](https://github.com/bashonly), [iw0nderhow](https://github.com/iw0nderhow)
* [extractor/bandcamp] Add `album_artist` by [stelcodes](https://github.com/stelcodes)
* [extractor/bilibili] Fix `--no-playlist` for anthology
* [extractor/bilibili] Improve `_VALID_URL` by [skbeh](https://github.com/skbeh)
* [extractor/biliintl:series] Make partial download of series faster
* [extractor/BiliLive] Fix extractor
* [extractor/brightcove] Add `BrightcoveNewBaseIE` and fix embed extraction
* [extractor/cda] Support premium and misc improvements by [selfisekai](https://github.com/selfisekai)
* [extractor/ciscowebex] Support password-protected videos by [damianoamatruda](https://github.com/damianoamatruda)
* [extractor/curiositystream] Fix auth by [mnn](https://github.com/mnn)
* [extractor/embedly] Handle vimeo embeds
* [extractor/fifa] Fix Preplay extraction by [dirkf](https://github.com/dirkf)
* [extractor/foxsports] Fix extractor by [bashonly](https://github.com/bashonly)
* [extractor/gronkh] Fix `_VALID_URL` by [muddi900](https://github.com/muddi900)
* [extractor/hotstar] Improve format metadata
* [extractor/iqiyi] Fix `Iq` JS regex by [bashonly](https://github.com/bashonly)
* [extractor/la7] Improve extractor by [nixxo](https://github.com/nixxo)
* [extractor/mediaset] Better embed detection and error messages by [nixxo](https://github.com/nixxo)
* [extractor/mixch] Support `--wait-for-video`
* [extractor/naver] Improve `_VALID_URL` for `NaverNowIE` by [bashonly](https://github.com/bashonly)
* [extractor/naver] Treat fan subtitles as separate language
* [extractor/netverse] Extract comments by [HobbyistDev](https://github.com/HobbyistDev)
* [extractor/nosnl] Add support for /video by [HobbyistDev](https://github.com/HobbyistDev)
* [extractor/odnoklassniki] Extract subtitles by [bashonly](https://github.com/bashonly)
* [extractor/pinterest] Fix extractor by [bashonly](https://github.com/bashonly)
* [extractor/plutotv] Fix videos with non-zero start by [digitall](https://github.com/digitall)
* [extractor/polskieradio] Adapt to next.js redesigns by [selfisekai](https://github.com/selfisekai)
* [extractor/reddit] Add vcodec to fallback format by [chengzhicn](https://github.com/chengzhicn)
* [extractor/reddit] Extract crossposted media by [bashonly](https://github.com/bashonly)
* [extractor/reddit] Extract video embeds in text posts by [bashonly](https://github.com/bashonly)
* [extractor/rutube] Support private videos by [mexus](https://github.com/mexus)
* [extractor/sibnet] Separate from VKIE
* [extractor/slideslive] Fix extractor by [Grub4K](https://github.com/Grub4K), [bashonly](https://github.com/bashonly)
* [extractor/slideslive] Support embeds and slides by [Grub4K](https://github.com/Grub4K), [bashonly](https://github.com/bashonly), [pukkandan](https://github.com/pukkandan)
* [extractor/soundcloud] Support user permalink by [nosoop](https://github.com/nosoop)
* [extractor/spankbang] Fix extractor by [JChris246](https://github.com/JChris246)
* [extractor/stv] Detect DRM
* [extractor/swearnet] Fix description bug
* [extractor/tencent] Fix geo-restricted video by [elyse0](https://github.com/elyse0)
* [extractor/tiktok] Fix subs, `DouyinIE`, improve `_VALID_URL` by [bashonly](https://github.com/bashonly)
* [extractor/tiktok] Update `_VALID_URL`, add `api_hostname` arg by [bashonly](https://github.com/bashonly)
* [extractor/tiktok] Update API hostname by [redraskal](https://github.com/redraskal)
* [extractor/twitcasting] Fix videos with password by [Spicadox](https://github.com/Spicadox), [bashonly](https://github.com/bashonly)
* [extractor/twitter] Heed `--no-playlist` for multi-video tweets by [Grub4K](https://github.com/Grub4K), [bashonly](https://github.com/bashonly)
* [extractor/twitter] Refresh guest token when expired by [Grub4K](https://github.com/Grub4K), [bashonly](https://github.com/bashonly)
* [extractor/twitter:spaces] Add `Referer` to m3u8 by [nixxo](https://github.com/nixxo)
* [extractor/udemy] Fix lectures that have no URL and detect DRM
* [extractor/unsupported] Add more URLs
* [extractor/urplay] Support for audio-only formats by [barsnick](https://github.com/barsnick)
* [extractor/wistia] Improve extension detection by [Grub4k](https://github.com/Grub4k), [bashonly](https://github.com/bashonly), [pukkandan](https://github.com/pukkandan)
* [extractor/yle_areena] Support restricted videos by [docbender](https://github.com/docbender)
* [extractor/youku] Fix extractor by [KurtBestor](https://github.com/KurtBestor)
* [extractor/youporn] Fix metadata by [marieell](https://github.com/marieell)
* [extractor/redgifs] Fix bug in [8c188d5](https://github.com/yt-dlp/yt-dlp/commit/8c188d5d09177ed213a05c900d3523867c5897fd)
### 2022.11.11 ### 2022.11.11
+19 -2
View File
@@ -8,6 +8,7 @@ You can also find lists of all [contributors of yt-dlp](CONTRIBUTORS) and [autho
## [pukkandan](https://github.com/pukkandan) ## [pukkandan](https://github.com/pukkandan)
[![ko-fi](https://img.shields.io/badge/_-Ko--fi-red.svg?logo=kofi&labelColor=555555&style=for-the-badge)](https://ko-fi.com/pukkandan) [![ko-fi](https://img.shields.io/badge/_-Ko--fi-red.svg?logo=kofi&labelColor=555555&style=for-the-badge)](https://ko-fi.com/pukkandan)
[![gh-sponsor](https://img.shields.io/badge/_-Github-red.svg?logo=github&labelColor=555555&style=for-the-badge)](https://github.com/sponsors/pukkandan)
* Owner of the fork * Owner of the fork
@@ -25,8 +26,9 @@ You can also find lists of all [contributors of yt-dlp](CONTRIBUTORS) and [autho
## [coletdjnz](https://github.com/coletdjnz) ## [coletdjnz](https://github.com/coletdjnz)
[![gh-sponsor](https://img.shields.io/badge/_-Sponsor-red.svg?logo=githubsponsors&labelColor=555555&style=for-the-badge)](https://github.com/sponsors/coletdjnz) [![gh-sponsor](https://img.shields.io/badge/_-Github-red.svg?logo=github&labelColor=555555&style=for-the-badge)](https://github.com/sponsors/coletdjnz)
* Improved plugin architecture
* YouTube improvements including: age-gate bypass, private playlists, multiple-clients (to avoid throttling) and a lot of under-the-hood improvements * YouTube improvements including: age-gate bypass, private playlists, multiple-clients (to avoid throttling) and a lot of under-the-hood improvements
* Added support for new websites YoutubeWebArchive, MainStreaming, PRX, nzherald, Mediaklikk, StarTV etc * Added support for new websites YoutubeWebArchive, MainStreaming, PRX, nzherald, Mediaklikk, StarTV etc
* Improved/fixed support for Patreon, panopto, gfycat, itv, pbs, SouthParkDE etc * Improved/fixed support for Patreon, panopto, gfycat, itv, pbs, SouthParkDE etc
@@ -42,7 +44,7 @@ You can also find lists of all [contributors of yt-dlp](CONTRIBUTORS) and [autho
* Improved/fixed support for HiDive, HotStar, Hungama, LBRY, LinkedInLearning, Mxplayer, SonyLiv, TV2, Vimeo, VLive etc * Improved/fixed support for HiDive, HotStar, Hungama, LBRY, LinkedInLearning, Mxplayer, SonyLiv, TV2, Vimeo, VLive etc
## [Lesmiscore](https://github.com/Lesmiscore) (nao20010128nao) ## [Lesmiscore](https://github.com/Lesmiscore) <sub><sup>(nao20010128nao)</sup></sub>
**Bitcoin**: bc1qfd02r007cutfdjwjmyy9w23rjvtls6ncve7r3s **Bitcoin**: bc1qfd02r007cutfdjwjmyy9w23rjvtls6ncve7r3s
**Monacoin**: mona1q3tf7dzvshrhfe3md379xtvt2n22duhglv5dskr **Monacoin**: mona1q3tf7dzvshrhfe3md379xtvt2n22duhglv5dskr
@@ -50,3 +52,18 @@ You can also find lists of all [contributors of yt-dlp](CONTRIBUTORS) and [autho
* Download live from start to end for YouTube * Download live from start to end for YouTube
* Added support for new websites AbemaTV, mildom, PixivSketch, skeb, radiko, voicy, mirrativ, openrec, whowatch, damtomo, 17.live, mixch etc * Added support for new websites AbemaTV, mildom, PixivSketch, skeb, radiko, voicy, mirrativ, openrec, whowatch, damtomo, 17.live, mixch etc
* Improved/fixed support for fc2, YahooJapanNews, tver, iwara etc * Improved/fixed support for fc2, YahooJapanNews, tver, iwara etc
## [bashonly](https://github.com/bashonly)
* `--cookies-from-browser` support for Firefox containers
* Added support for new websites Genius, Kick, NBCStations, Triller, VideoKen etc
* Improved/fixed support for Anvato, Brightcove, Instagram, ParamountPlus, Reddit, SlidesLive, TikTok, Twitter, Vimeo etc
## [Grub4K](https://github.com/Grub4K)
[![ko-fi](https://img.shields.io/badge/_-Ko--fi-red.svg?logo=kofi&labelColor=555555&style=for-the-badge)](https://ko-fi.com/Grub4K) [![gh-sponsor](https://img.shields.io/badge/_-Github-red.svg?logo=github&labelColor=555555&style=for-the-badge)](https://github.com/sponsors/Grub4K)
* Rework internals like `traverse_obj`, various core refactors and bugs fixes
* Helped fix crunchyroll, Twitter, wrestleuniverse, wistia, slideslive etc
+3 -3
View File
@@ -17,8 +17,8 @@ pypi-files: AUTHORS Changelog.md LICENSE README.md README.txt supportedsites \
clean-test: clean-test:
rm -rf test/testdata/sigs/player-*.js tmp/ *.annotations.xml *.aria2 *.description *.dump *.frag \ rm -rf test/testdata/sigs/player-*.js tmp/ *.annotations.xml *.aria2 *.description *.dump *.frag \
*.frag.aria2 *.frag.urls *.info.json *.live_chat.json *.meta *.part* *.tmp *.temp *.unknown_video *.ytdl \ *.frag.aria2 *.frag.urls *.info.json *.live_chat.json *.meta *.part* *.tmp *.temp *.unknown_video *.ytdl \
*.3gp *.ape *.ass *.avi *.desktop *.f4v *.flac *.flv *.jpeg *.jpg *.m4a *.m4v *.mhtml *.mkv *.mov *.mp3 *.mp4 \ *.3gp *.ape *.ass *.avi *.desktop *.f4v *.flac *.flv *.gif *.jpeg *.jpg *.m4a *.m4v *.mhtml *.mkv *.mov *.mp3 \
*.mpga *.oga *.ogg *.opus *.png *.sbv *.srt *.swf *.swp *.tt *.ttml *.url *.vtt *.wav *.webloc *.webm *.webp *.mp4 *.mpga *.oga *.ogg *.opus *.png *.sbv *.srt *.swf *.swp *.tt *.ttml *.url *.vtt *.wav *.webloc *.webm *.webp
clean-dist: clean-dist:
rm -rf yt-dlp.1.temp.md yt-dlp.1 README.txt MANIFEST build/ dist/ .coverage cover/ yt-dlp.tar.gz completions/ \ rm -rf yt-dlp.1.temp.md yt-dlp.1 README.txt MANIFEST build/ dist/ .coverage cover/ yt-dlp.tar.gz completions/ \
yt_dlp/extractor/lazy_extractors.py *.spec CONTRIBUTING.md.tmp yt-dlp yt-dlp.exe yt_dlp.egg-info/ AUTHORS .mailmap yt_dlp/extractor/lazy_extractors.py *.spec CONTRIBUTING.md.tmp yt-dlp yt-dlp.exe yt_dlp.egg-info/ AUTHORS .mailmap
@@ -74,7 +74,7 @@ offlinetest: codetest
$(PYTHON) -m pytest -k "not download" $(PYTHON) -m pytest -k "not download"
# XXX: This is hard to maintain # XXX: This is hard to maintain
CODE_FOLDERS = yt_dlp yt_dlp/downloader yt_dlp/extractor yt_dlp/postprocessor yt_dlp/compat CODE_FOLDERS = yt_dlp yt_dlp/downloader yt_dlp/extractor yt_dlp/postprocessor yt_dlp/compat yt_dlp/dependencies
yt-dlp: yt_dlp/*.py yt_dlp/*/*.py yt-dlp: yt_dlp/*.py yt_dlp/*/*.py
mkdir -p zip mkdir -p zip
for d in $(CODE_FOLDERS) ; do \ for d in $(CODE_FOLDERS) ; do \
+153 -71
View File
@@ -10,7 +10,7 @@
[![Discord](https://img.shields.io/discord/807245652072857610?color=blue&labelColor=555555&label=&logo=discord&style=for-the-badge)](https://discord.gg/H5MNcFW63r "Discord") [![Discord](https://img.shields.io/discord/807245652072857610?color=blue&labelColor=555555&label=&logo=discord&style=for-the-badge)](https://discord.gg/H5MNcFW63r "Discord")
[![Supported Sites](https://img.shields.io/badge/-Supported_Sites-brightgreen.svg?style=for-the-badge)](supportedsites.md "Supported Sites") [![Supported Sites](https://img.shields.io/badge/-Supported_Sites-brightgreen.svg?style=for-the-badge)](supportedsites.md "Supported Sites")
[![License: Unlicense](https://img.shields.io/badge/-Unlicense-blue.svg?style=for-the-badge)](LICENSE "License") [![License: Unlicense](https://img.shields.io/badge/-Unlicense-blue.svg?style=for-the-badge)](LICENSE "License")
[![CI Status](https://img.shields.io/github/workflow/status/yt-dlp/yt-dlp/Core%20Tests/master?label=Tests&style=for-the-badge)](https://github.com/yt-dlp/yt-dlp/actions "CI Status") [![CI Status](https://img.shields.io/github/actions/workflow/status/yt-dlp/yt-dlp/core.yml?branch=master&label=Tests&style=for-the-badge)](https://github.com/yt-dlp/yt-dlp/actions "CI Status")
[![Commits](https://img.shields.io/github/commit-activity/m/yt-dlp/yt-dlp?label=commits&style=for-the-badge)](https://github.com/yt-dlp/yt-dlp/commits "Commit History") [![Commits](https://img.shields.io/github/commit-activity/m/yt-dlp/yt-dlp?label=commits&style=for-the-badge)](https://github.com/yt-dlp/yt-dlp/commits "Commit History")
[![Last Commit](https://img.shields.io/github/last-commit/yt-dlp/yt-dlp/master?label=&style=for-the-badge&display_timestamp=committer)](https://github.com/yt-dlp/yt-dlp/commits "Commit History") [![Last Commit](https://img.shields.io/github/last-commit/yt-dlp/yt-dlp/master?label=&style=for-the-badge&display_timestamp=committer)](https://github.com/yt-dlp/yt-dlp/commits "Commit History")
@@ -61,6 +61,8 @@ yt-dlp is a [youtube-dl](https://github.com/ytdl-org/youtube-dl) fork based on t
* [Modifying metadata examples](#modifying-metadata-examples) * [Modifying metadata examples](#modifying-metadata-examples)
* [EXTRACTOR ARGUMENTS](#extractor-arguments) * [EXTRACTOR ARGUMENTS](#extractor-arguments)
* [PLUGINS](#plugins) * [PLUGINS](#plugins)
* [Installing Plugins](#installing-plugins)
* [Developing Plugins](#developing-plugins)
* [EMBEDDING YT-DLP](#embedding-yt-dlp) * [EMBEDDING YT-DLP](#embedding-yt-dlp)
* [Embedding examples](#embedding-examples) * [Embedding examples](#embedding-examples)
* [DEPRECATED OPTIONS](#deprecated-options) * [DEPRECATED OPTIONS](#deprecated-options)
@@ -74,13 +76,13 @@ yt-dlp is a [youtube-dl](https://github.com/ytdl-org/youtube-dl) fork based on t
# NEW FEATURES # NEW FEATURES
* Merged with **youtube-dl v2021.12.17+ [commit/de39d12](https://github.com/ytdl-org/youtube-dl/commit/de39d128)** <!--([exceptions](https://github.com/yt-dlp/yt-dlp/issues/21))--> and **youtube-dlc v2020.11.11-3+ [commit/f9401f2](https://github.com/blackjack4494/yt-dlc/commit/f9401f2a91987068139c5f757b12fc711d4c0cee)**: You get all the features and patches of [youtube-dlc](https://github.com/blackjack4494/yt-dlc) in addition to the latest [youtube-dl](https://github.com/ytdl-org/youtube-dl) * Merged with **youtube-dl v2021.12.17+ [commit/2dd6c6e](https://github.com/ytdl-org/youtube-dl/commit/2dd6c6e)** ([exceptions](https://github.com/yt-dlp/yt-dlp/issues/21)) and **youtube-dlc v2020.11.11-3+ [commit/f9401f2](https://github.com/blackjack4494/yt-dlc/commit/f9401f2a91987068139c5f757b12fc711d4c0cee)**: You get all the features and patches of [youtube-dlc](https://github.com/blackjack4494/yt-dlc) in addition to the latest [youtube-dl](https://github.com/ytdl-org/youtube-dl)
* **[SponsorBlock Integration](#sponsorblock-options)**: You can mark/remove sponsor sections in YouTube videos by utilizing the [SponsorBlock](https://sponsor.ajay.app) API * **[SponsorBlock Integration](#sponsorblock-options)**: You can mark/remove sponsor sections in YouTube videos by utilizing the [SponsorBlock](https://sponsor.ajay.app) API
* **[Format Sorting](#sorting-formats)**: The default format sorting options have been changed so that higher resolution and better codecs will be now preferred instead of simply using larger bitrate. Furthermore, you can now specify the sort order using `-S`. This allows for much easier format selection than what is possible by simply using `--format` ([examples](#format-selection-examples)) * **[Format Sorting](#sorting-formats)**: The default format sorting options have been changed so that higher resolution and better codecs will be now preferred instead of simply using larger bitrate. Furthermore, you can now specify the sort order using `-S`. This allows for much easier format selection than what is possible by simply using `--format` ([examples](#format-selection-examples))
* **Merged with animelover1984/youtube-dl**: You get most of the features and improvements from [animelover1984/youtube-dl](https://github.com/animelover1984/youtube-dl) including `--write-comments`, `BiliBiliSearch`, `BilibiliChannel`, Embedding thumbnail in mp4/ogg/opus, playlist infojson etc. Note that the NicoNico livestreams are not available. See [#31](https://github.com/yt-dlp/yt-dlp/pull/31) for details. * **Merged with animelover1984/youtube-dl**: You get most of the features and improvements from [animelover1984/youtube-dl](https://github.com/animelover1984/youtube-dl) including `--write-comments`, `BiliBiliSearch`, `BilibiliChannel`, Embedding thumbnail in mp4/ogg/opus, playlist infojson etc. Note that NicoNico livestreams are not available. See [#31](https://github.com/yt-dlp/yt-dlp/pull/31) for details.
* **YouTube improvements**: * **YouTube improvements**:
* Supports Clips, Stories (`ytstories:<channel UCID>`), Search (including filters)**\***, YouTube Music Search, Channel-specific search, Search prefixes (`ytsearch:`, `ytsearchdate:`)**\***, Mixes, YouTube Music Albums/Channels ([except self-uploaded music](https://github.com/yt-dlp/yt-dlp/issues/723)), and Feeds (`:ytfav`, `:ytwatchlater`, `:ytsubs`, `:ythistory`, `:ytrec`, `:ytnotif`) * Supports Clips, Stories (`ytstories:<channel UCID>`), Search (including filters)**\***, YouTube Music Search, Channel-specific search, Search prefixes (`ytsearch:`, `ytsearchdate:`)**\***, Mixes, YouTube Music Albums/Channels ([except self-uploaded music](https://github.com/yt-dlp/yt-dlp/issues/723)), and Feeds (`:ytfav`, `:ytwatchlater`, `:ytsubs`, `:ythistory`, `:ytrec`, `:ytnotif`)
@@ -151,12 +153,15 @@ Some of yt-dlp's default options are different from that of youtube-dl and youtu
* When `--embed-subs` and `--write-subs` are used together, the subtitles are written to disk and also embedded in the media file. You can use just `--embed-subs` to embed the subs and automatically delete the separate file. See [#630 (comment)](https://github.com/yt-dlp/yt-dlp/issues/630#issuecomment-893659460) for more info. `--compat-options no-keep-subs` can be used to revert this * When `--embed-subs` and `--write-subs` are used together, the subtitles are written to disk and also embedded in the media file. You can use just `--embed-subs` to embed the subs and automatically delete the separate file. See [#630 (comment)](https://github.com/yt-dlp/yt-dlp/issues/630#issuecomment-893659460) for more info. `--compat-options no-keep-subs` can be used to revert this
* `certifi` will be used for SSL root certificates, if installed. If you want to use system certificates (e.g. self-signed), use `--compat-options no-certifi` * `certifi` will be used for SSL root certificates, if installed. If you want to use system certificates (e.g. self-signed), use `--compat-options no-certifi`
* yt-dlp's sanitization of invalid characters in filenames is different/smarter than in youtube-dl. You can use `--compat-options filename-sanitization` to revert to youtube-dl's behavior * yt-dlp's sanitization of invalid characters in filenames is different/smarter than in youtube-dl. You can use `--compat-options filename-sanitization` to revert to youtube-dl's behavior
* yt-dlp tries to parse the external downloader outputs into the standard progress output if possible (Currently implemented: [~~aria2c~~](https://github.com/yt-dlp/yt-dlp/issues/5931)). You can use `--compat-options no-external-downloader-progress` to get the downloader output as-is
For ease of use, a few more compat options are available: For ease of use, a few more compat options are available:
* `--compat-options all`: Use all compat options (Do NOT use) * `--compat-options all`: Use all compat options (Do NOT use)
* `--compat-options youtube-dl`: Same as `--compat-options all,-multistreams` * `--compat-options youtube-dl`: Same as `--compat-options all,-multistreams`
* `--compat-options youtube-dlc`: Same as `--compat-options all,-no-live-chat,-no-youtube-channel-redirect` * `--compat-options youtube-dlc`: Same as `--compat-options all,-no-live-chat,-no-youtube-channel-redirect`
* `--compat-options 2021`: Same as `--compat-options 2022,no-certifi,filename-sanitization,no-youtube-prefer-utc-upload-date`
* `--compat-options 2022`: Same as `--compat-options no-external-downloader-progress`. Use this to enable all future compat options
# INSTALLATION # INSTALLATION
@@ -179,7 +184,7 @@ You can use `yt-dlp -U` to update if you are [using the release binaries](#relea
If you [installed with PIP](https://github.com/yt-dlp/yt-dlp/wiki/Installation#with-pip), simply re-run the same command that was used to install the program If you [installed with PIP](https://github.com/yt-dlp/yt-dlp/wiki/Installation#with-pip), simply re-run the same command that was used to install the program
For other third-party package managers, see [the wiki](https://github.com/yt-dlp/yt-dlp/wiki/Installation) or refer their documentation For other third-party package managers, see [the wiki](https://github.com/yt-dlp/yt-dlp/wiki/Installation#third-party-package-managers) or refer their documentation
<!-- MANPAGE: BEGIN EXCLUDED SECTION --> <!-- MANPAGE: BEGIN EXCLUDED SECTION -->
@@ -217,7 +222,7 @@ File|Description
<!-- MANPAGE: END EXCLUDED SECTION --> <!-- MANPAGE: END EXCLUDED SECTION -->
Note: The manpages, shell completion files etc. are available in the [source tarball](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp.tar.gz) **Note**: The manpages, shell completion files etc. are available in the [source tarball](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp.tar.gz)
## DEPENDENCIES ## DEPENDENCIES
Python versions 3.7+ (CPython and PyPy) are supported. Other versions and implementations may or may not work correctly. Python versions 3.7+ (CPython and PyPy) are supported. Other versions and implementations may or may not work correctly.
@@ -233,8 +238,9 @@ While all the other dependencies are optional, `ffmpeg` and `ffprobe` are highly
* [**ffmpeg** and **ffprobe**](https://www.ffmpeg.org) - Required for [merging separate video and audio files](#format-selection) as well as for various [post-processing](#post-processing-options) tasks. License [depends on the build](https://www.ffmpeg.org/legal.html) * [**ffmpeg** and **ffprobe**](https://www.ffmpeg.org) - Required for [merging separate video and audio files](#format-selection) as well as for various [post-processing](#post-processing-options) tasks. License [depends on the build](https://www.ffmpeg.org/legal.html)
<!-- TODO: ffmpeg has merged this patch. Remove this note once there is new release --> There are bugs in ffmpeg that causes various issues when used alongside yt-dlp. Since ffmpeg is such an important dependency, we provide [custom builds](https://github.com/yt-dlp/FFmpeg-Builds#ffmpeg-static-auto-builds) with patches for some of these issues at [yt-dlp/FFmpeg-Builds](https://github.com/yt-dlp/FFmpeg-Builds). See [the readme](https://github.com/yt-dlp/FFmpeg-Builds#patches-applied) for details on the specific issues solved by these builds
**Note**: There are some regressions in newer ffmpeg versions that causes various issues when used alongside yt-dlp. Since ffmpeg is such an important dependency, we provide [custom builds](https://github.com/yt-dlp/FFmpeg-Builds#ffmpeg-static-auto-builds) with patches for these issues at [yt-dlp/FFmpeg-Builds](https://github.com/yt-dlp/FFmpeg-Builds). See [the readme](https://github.com/yt-dlp/FFmpeg-Builds#patches-applied) for details on the specific issues solved by these builds
**Important**: What you need is ffmpeg *binary*, **NOT** [the python package of the same name](https://pypi.org/project/ffmpeg)
### Networking ### Networking
* [**certifi**](https://github.com/certifi/python-certifi)\* - Provides Mozilla's root certificate bundle. Licensed under [MPLv2](https://github.com/certifi/python-certifi/blob/master/LICENSE) * [**certifi**](https://github.com/certifi/python-certifi)\* - Provides Mozilla's root certificate bundle. Licensed under [MPLv2](https://github.com/certifi/python-certifi/blob/master/LICENSE)
@@ -281,7 +287,7 @@ On some systems, you may need to use `py` or `python` instead of `python3`.
`pyinst.py` accepts any arguments that can be passed to `pyinstaller`, such as `--onefile/-F` or `--onedir/-D`, which is further [documented here](https://pyinstaller.org/en/stable/usage.html#what-to-generate). `pyinst.py` accepts any arguments that can be passed to `pyinstaller`, such as `--onefile/-F` or `--onedir/-D`, which is further [documented here](https://pyinstaller.org/en/stable/usage.html#what-to-generate).
Note that pyinstaller with versions below 4.4 [do not support](https://github.com/pyinstaller/pyinstaller#requirements-and-tested-platforms) Python installed from the Windows store without using a virtual environment. **Note**: Pyinstaller versions below 4.4 [do not support](https://github.com/pyinstaller/pyinstaller#requirements-and-tested-platforms) Python installed from the Windows store without using a virtual environment.
**Important**: Running `pyinstaller` directly **without** using `pyinst.py` is **not** officially supported. This may or may not work correctly. **Important**: Running `pyinstaller` directly **without** using `pyinst.py` is **not** officially supported. This may or may not work correctly.
@@ -414,6 +420,8 @@ You can also fork the project on GitHub and run your fork's [build workflow](.gi
--source-address IP Client-side IP address to bind to --source-address IP Client-side IP address to bind to
-4, --force-ipv4 Make all connections via IPv4 -4, --force-ipv4 Make all connections via IPv4
-6, --force-ipv6 Make all connections via IPv6 -6, --force-ipv6 Make all connections via IPv6
--enable-file-urls Enable file:// URLs. This is disabled by
default for security reasons.
## Geo-restriction: ## Geo-restriction:
--geo-verification-proxy URL Use this proxy to verify the IP address for --geo-verification-proxy URL Use this proxy to verify the IP address for
@@ -432,23 +440,25 @@ You can also fork the project on GitHub and run your fork's [build workflow](.gi
explicitly provided IP block in CIDR notation explicitly provided IP block in CIDR notation
## Video Selection: ## Video Selection:
-I, --playlist-items ITEM_SPEC Comma separated playlist_index of the videos -I, --playlist-items ITEM_SPEC Comma separated playlist_index of the items
to download. You can specify a range using to download. You can specify a range using
"[START]:[STOP][:STEP]". For backward "[START]:[STOP][:STEP]". For backward
compatibility, START-STOP is also supported. compatibility, START-STOP is also supported.
Use negative indices to count from the right Use negative indices to count from the right
and negative STEP to download in reverse and negative STEP to download in reverse
order. E.g. "-I 1:3,7,-5::2" used on a order. E.g. "-I 1:3,7,-5::2" used on a
playlist of size 15 will download the videos playlist of size 15 will download the items
at index 1,2,3,7,11,13,15 at index 1,2,3,7,11,13,15
--min-filesize SIZE Do not download any videos smaller than --min-filesize SIZE Abort download if filesize is smaller than
SIZE, e.g. 50k or 44.6M
--max-filesize SIZE Abort download if filesize is larger than
SIZE, e.g. 50k or 44.6M SIZE, e.g. 50k or 44.6M
--max-filesize SIZE Do not download any videos larger than SIZE,
e.g. 50k or 44.6M
--date DATE Download only videos uploaded on this date. --date DATE Download only videos uploaded on this date.
The date can be "YYYYMMDD" or in the format The date can be "YYYYMMDD" or in the format
[now|today|yesterday][-N[day|week|month|year]]. [now|today|yesterday][-N[day|week|month|year]].
E.g. --date today-2weeks E.g. "--date today-2weeks" downloads
only videos uploaded on the same day two
weeks ago
--datebefore DATE Download only videos uploaded on or before --datebefore DATE Download only videos uploaded on or before
this date. The date formats accepted is the this date. The date formats accepted is the
same as --date same as --date
@@ -491,9 +501,9 @@ You can also fork the project on GitHub and run your fork's [build workflow](.gi
a file that is in the archive a file that is in the archive
--break-on-reject Stop the download process when encountering --break-on-reject Stop the download process when encountering
a file that has been filtered out a file that has been filtered out
--break-per-input --break-on-existing, --break-on-reject, --break-per-input Alters --max-downloads, --break-on-existing,
--max-downloads, and autonumber resets per --break-on-reject, and autonumber to reset
input URL per input URL
--no-break-per-input --break-on-existing and similar options --no-break-per-input --break-on-existing and similar options
terminates the entire download queue terminates the entire download queue
--skip-playlist-after-errors N Number of allowed failures until the rest of --skip-playlist-after-errors N Number of allowed failures until the rest of
@@ -525,8 +535,8 @@ You can also fork the project on GitHub and run your fork's [build workflow](.gi
linear=1::2 --retry-sleep fragment:exp=1:20 linear=1::2 --retry-sleep fragment:exp=1:20
--skip-unavailable-fragments Skip unavailable fragments for DASH, --skip-unavailable-fragments Skip unavailable fragments for DASH,
hlsnative and ISM downloads (default) hlsnative and ISM downloads (default)
(Alias: --no-abort-on-unavailable-fragment) (Alias: --no-abort-on-unavailable-fragments)
--abort-on-unavailable-fragment --abort-on-unavailable-fragments
Abort download if a fragment is unavailable Abort download if a fragment is unavailable
(Alias: --no-skip-unavailable-fragments) (Alias: --no-skip-unavailable-fragments)
--keep-fragments Keep downloaded fragments on disk after --keep-fragments Keep downloaded fragments on disk after
@@ -725,7 +735,7 @@ You can also fork the project on GitHub and run your fork's [build workflow](.gi
screen, optionally prefixed with when to screen, optionally prefixed with when to
print it, separated by a ":". Supported print it, separated by a ":". Supported
values of "WHEN" are the same as that of values of "WHEN" are the same as that of
--use-postprocessor, and "video" (default). --use-postprocessor (default: video).
Implies --quiet. Implies --simulate unless Implies --quiet. Implies --simulate unless
--no-simulate or later stages of WHEN are --no-simulate or later stages of WHEN are
used. This option can be used multiple times used. This option can be used multiple times
@@ -778,7 +788,7 @@ You can also fork the project on GitHub and run your fork's [build workflow](.gi
--prefer-insecure Use an unencrypted connection to retrieve --prefer-insecure Use an unencrypted connection to retrieve
information about the video (Currently information about the video (Currently
supported only for YouTube) supported only for YouTube)
--add-header FIELD:VALUE Specify a custom HTTP header and its value, --add-headers FIELD:VALUE Specify a custom HTTP header and its value,
separated by a colon ":". You can use this separated by a colon ":". You can use this
option multiple times option multiple times
--bidi-workaround Work around terminals that lack --bidi-workaround Work around terminals that lack
@@ -893,11 +903,11 @@ You can also fork the project on GitHub and run your fork's [build workflow](.gi
specific bitrate like 128K (default 5) specific bitrate like 128K (default 5)
--remux-video FORMAT Remux the video into another container if --remux-video FORMAT Remux the video into another container if
necessary (currently supported: avi, flv, necessary (currently supported: avi, flv,
mkv, mov, mp4, webm, aac, aiff, alac, flac, gif, mkv, mov, mp4, webm, aac, aiff, alac,
m4a, mka, mp3, ogg, opus, vorbis, wav). If flac, m4a, mka, mp3, ogg, opus, vorbis,
target container does not support the wav). If target container does not support
video/audio codec, remuxing will fail. You the video/audio codec, remuxing will fail.
can specify multiple rules; e.g. You can specify multiple rules; e.g.
"aac>m4a/mov>mp4/mkv" will remux aac to m4a, "aac>m4a/mov>mp4/mkv" will remux aac to m4a,
mov to mp4 and anything else to mkv mov to mp4 and anything else to mkv
--recode-video FORMAT Re-encode the video into another format if --recode-video FORMAT Re-encode the video into another format if
@@ -952,13 +962,18 @@ You can also fork the project on GitHub and run your fork's [build workflow](.gi
mkv/mka video files mkv/mka video files
--no-embed-info-json Do not embed the infojson as an attachment --no-embed-info-json Do not embed the infojson as an attachment
to the video file to the video file
--parse-metadata FROM:TO Parse additional metadata like title/artist --parse-metadata [WHEN:]FROM:TO
Parse additional metadata like title/artist
from other fields; see "MODIFYING METADATA" from other fields; see "MODIFYING METADATA"
for details for details. Supported values of "WHEN" are
--replace-in-metadata FIELDS REGEX REPLACE the same as that of --use-postprocessor
(default: pre_process)
--replace-in-metadata [WHEN:]FIELDS REGEX REPLACE
Replace text in a metadata field using the Replace text in a metadata field using the
given regex. This option can be used given regex. This option can be used
multiple times multiple times. Supported values of "WHEN"
are the same as that of --use-postprocessor
(default: pre_process)
--xattrs Write metadata to the video file's xattrs --xattrs Write metadata to the video file's xattrs
(using dublin core and xdg standards) (using dublin core and xdg standards)
--concat-playlist POLICY Concatenate videos in a playlist. One of --concat-playlist POLICY Concatenate videos in a playlist. One of
@@ -979,18 +994,18 @@ You can also fork the project on GitHub and run your fork's [build workflow](.gi
--ffmpeg-location PATH Location of the ffmpeg binary; either the --ffmpeg-location PATH Location of the ffmpeg binary; either the
path to the binary or its containing directory path to the binary or its containing directory
--exec [WHEN:]CMD Execute a command, optionally prefixed with --exec [WHEN:]CMD Execute a command, optionally prefixed with
when to execute it (after_move if when to execute it, separated by a ":".
unspecified), separated by a ":". Supported Supported values of "WHEN" are the same as
values of "WHEN" are the same as that of that of --use-postprocessor (default:
--use-postprocessor. Same syntax as the after_move). Same syntax as the output
output template can be used to pass any template can be used to pass any field as
field as arguments to the command. After arguments to the command. After download, an
download, an additional field "filepath" additional field "filepath" that contains
that contains the final path of the the final path of the downloaded file is
downloaded file is also available, and if no also available, and if no fields are passed,
fields are passed, %(filepath)q is appended %(filepath,_filename|)q is appended to the
to the end of the command. This option can end of the command. This option can be used
be used multiple times multiple times
--no-exec Remove any previously defined --exec --no-exec Remove any previously defined --exec
--convert-subs FORMAT Convert the subtitles to another format --convert-subs FORMAT Convert the subtitles to another format
(currently supported: ass, lrc, srt, vtt) (currently supported: ass, lrc, srt, vtt)
@@ -1028,14 +1043,16 @@ You can also fork the project on GitHub and run your fork's [build workflow](.gi
postprocessor is invoked. It can be one of postprocessor is invoked. It can be one of
"pre_process" (after video extraction), "pre_process" (after video extraction),
"after_filter" (after video passes filter), "after_filter" (after video passes filter),
"before_dl" (before each video download), "video" (after --format; before
"post_process" (after each video download; --print/--output), "before_dl" (before each
default), "after_move" (after moving video video download), "post_process" (after each
file to it's final locations), "after_video" video download; default), "after_move"
(after downloading and processing all (after moving video file to it's final
formats of a video), or "playlist" (at end locations), "after_video" (after downloading
of playlist). This option can be used and processing all formats of a video), or
multiple times to add different postprocessors "playlist" (at end of playlist). This option
can be used multiple times to add different
postprocessors
## SponsorBlock Options: ## SponsorBlock Options:
Make chapter entries for, or remove various segments (sponsor, Make chapter entries for, or remove various segments (sponsor,
@@ -1046,10 +1063,10 @@ Make chapter entries for, or remove various segments (sponsor,
for, separated by commas. Available for, separated by commas. Available
categories are sponsor, intro, outro, categories are sponsor, intro, outro,
selfpromo, preview, filler, interaction, selfpromo, preview, filler, interaction,
music_offtopic, poi_highlight, chapter, all and music_offtopic, poi_highlight, chapter, all
default (=all). You can prefix the category and default (=all). You can prefix the
with a "-" to exclude it. See [1] for category with a "-" to exclude it. See [1]
description of the categories. E.g. for description of the categories. E.g.
--sponsorblock-mark all,-preview --sponsorblock-mark all,-preview
[1] https://wiki.sponsor.ajay.app/w/Segment_Categories [1] https://wiki.sponsor.ajay.app/w/Segment_Categories
--sponsorblock-remove CATS SponsorBlock categories to be removed from --sponsorblock-remove CATS SponsorBlock categories to be removed from
@@ -1058,7 +1075,7 @@ Make chapter entries for, or remove various segments (sponsor,
remove takes precedence. The syntax and remove takes precedence. The syntax and
available categories are the same as for available categories are the same as for
--sponsorblock-mark except that "default" --sponsorblock-mark except that "default"
refers to "all,-filler" and poi_highlight and refers to "all,-filler" and poi_highlight,
chapter are not available chapter are not available
--sponsorblock-chapter-title TEMPLATE --sponsorblock-chapter-title TEMPLATE
An output template for the title of the An output template for the title of the
@@ -1102,16 +1119,22 @@ You can configure yt-dlp by placing any supported command line option to a confi
* `yt-dlp.conf` in the home path given by `-P` * `yt-dlp.conf` in the home path given by `-P`
* If `-P` is not given, the current directory is searched * If `-P` is not given, the current directory is searched
1. **User Configuration**: 1. **User Configuration**:
* `${XDG_CONFIG_HOME}/yt-dlp/config` (recommended on Linux/macOS)
* `${XDG_CONFIG_HOME}/yt-dlp.conf` * `${XDG_CONFIG_HOME}/yt-dlp.conf`
* `${XDG_CONFIG_HOME}/yt-dlp/config` (recommended on Linux/macOS)
* `${XDG_CONFIG_HOME}/yt-dlp/config.txt`
* `${APPDATA}/yt-dlp.conf`
* `${APPDATA}/yt-dlp/config` (recommended on Windows) * `${APPDATA}/yt-dlp/config` (recommended on Windows)
* `${APPDATA}/yt-dlp/config.txt` * `${APPDATA}/yt-dlp/config.txt`
* `~/yt-dlp.conf` * `~/yt-dlp.conf`
* `~/yt-dlp.conf.txt` * `~/yt-dlp.conf.txt`
* `~/.yt-dlp/config`
* `~/.yt-dlp/config.txt`
See also: [Notes about environment variables](#notes-about-environment-variables) See also: [Notes about environment variables](#notes-about-environment-variables)
1. **System Configuration**: 1. **System Configuration**:
* `/etc/yt-dlp.conf` * `/etc/yt-dlp.conf`
* `/etc/yt-dlp/config`
* `/etc/yt-dlp/config.txt`
E.g. with the following configuration file yt-dlp will always extract the audio, not copy the mtime, use a proxy and save all videos under `YouTube` directory in your home directory: E.g. with the following configuration file yt-dlp will always extract the audio, not copy the mtime, use a proxy and save all videos under `YouTube` directory in your home directory:
``` ```
@@ -1130,7 +1153,7 @@ E.g. with the following configuration file yt-dlp will always extract the audio,
-o ~/YouTube/%(title)s.%(ext)s -o ~/YouTube/%(title)s.%(ext)s
``` ```
Note that options in configuration file are just the same options aka switches used in regular command line calls; thus there **must be no whitespace** after `-` or `--`, e.g. `-o` or `--proxy` but not `- o` or `-- proxy`. They must also be quoted when necessary as-if it were a UNIX shell. **Note**: Options in configuration file are just the same options aka switches used in regular command line calls; thus there **must be no whitespace** after `-` or `--`, e.g. `-o` or `--proxy` but not `- o` or `-- proxy`. They must also be quoted when necessary as-if it were a UNIX shell.
You can use `--ignore-config` if you want to disable all configuration files for a particular yt-dlp run. If `--ignore-config` is found inside any configuration file, no further configuration will be loaded. For example, having the option in the portable configuration file prevents loading of home, user, and system configurations. Additionally, (for backward compatibility) if `--ignore-config` is found inside the system configuration file, the user configuration is not loaded. You can use `--ignore-config` if you want to disable all configuration files for a particular yt-dlp run. If `--ignore-config` is found inside any configuration file, no further configuration will be loaded. For example, having the option in the portable configuration file prevents loading of home, user, and system configurations. Additionally, (for backward compatibility) if `--ignore-config` is found inside the system configuration file, the user configuration is not loaded.
@@ -1206,7 +1229,7 @@ Additionally, you can set different output templates for the various metadata fi
<a id="outtmpl-postprocess-note"></a> <a id="outtmpl-postprocess-note"></a>
Note: Due to post-processing (i.e. merging etc.), the actual output filename might differ. Use `--print after_move:filepath` to get the name after all post-processing is complete. **Note**: Due to post-processing (i.e. merging etc.), the actual output filename might differ. Use `--print after_move:filepath` to get the name after all post-processing is complete.
The available fields are: The available fields are:
@@ -1327,7 +1350,7 @@ Available only in `--sponsorblock-chapter-title`:
Each aforementioned sequence when referenced in an output template will be replaced by the actual value corresponding to the sequence name. E.g. for `-o %(title)s-%(id)s.%(ext)s` and an mp4 video with title `yt-dlp test video` and id `BaW_jenozKc`, this will result in a `yt-dlp test video-BaW_jenozKc.mp4` file created in the current directory. Each aforementioned sequence when referenced in an output template will be replaced by the actual value corresponding to the sequence name. E.g. for `-o %(title)s-%(id)s.%(ext)s` and an mp4 video with title `yt-dlp test video` and id `BaW_jenozKc`, this will result in a `yt-dlp test video-BaW_jenozKc.mp4` file created in the current directory.
Note that some of the sequences are not guaranteed to be present since they depend on the metadata obtained by a particular extractor. Such sequences will be replaced with placeholder value provided with `--output-na-placeholder` (`NA` by default). **Note**: Some of the sequences are not guaranteed to be present since they depend on the metadata obtained by a particular extractor. Such sequences will be replaced with placeholder value provided with `--output-na-placeholder` (`NA` by default).
**Tip**: Look at the `-j` output to identify which fields are available for the particular URL **Tip**: Look at the `-j` output to identify which fields are available for the particular URL
@@ -1442,6 +1465,7 @@ The following numeric meta fields can be used with comparisons `<`, `<=`, `>`, `
- `filesize_approx`: An estimate for the number of bytes - `filesize_approx`: An estimate for the number of bytes
- `width`: Width of the video, if known - `width`: Width of the video, if known
- `height`: Height of the video, if known - `height`: Height of the video, if known
- `aspect_ratio`: Aspect ratio of the video, if known
- `tbr`: Average bitrate of audio and video in KBit/s - `tbr`: Average bitrate of audio and video in KBit/s
- `abr`: Average audio bitrate in KBit/s - `abr`: Average audio bitrate in KBit/s
- `vbr`: Average video bitrate in KBit/s - `vbr`: Average video bitrate in KBit/s
@@ -1467,7 +1491,7 @@ Also filtering work for comparisons `=` (equals), `^=` (starts with), `$=` (ends
Any string comparison may be prefixed with negation `!` in order to produce an opposite comparison, e.g. `!*=` (does not contain). The comparand of a string comparison needs to be quoted with either double or single quotes if it contains spaces or special characters other than `._-`. Any string comparison may be prefixed with negation `!` in order to produce an opposite comparison, e.g. `!*=` (does not contain). The comparand of a string comparison needs to be quoted with either double or single quotes if it contains spaces or special characters other than `._-`.
Note that none of the aforementioned meta fields are guaranteed to be present since this solely depends on the metadata obtained by particular extractor, i.e. the metadata offered by the website. Any other field made available by the extractor can also be used for filtering. **Note**: None of the aforementioned meta fields are guaranteed to be present since this solely depends on the metadata obtained by particular extractor, i.e. the metadata offered by the website. Any other field made available by the extractor can also be used for filtering.
Formats for which the value is not known are excluded unless you put a question mark (`?`) after the operator. You can combine format filters, so `-f "[height<=?720][tbr>500]"` selects up to 720p videos (or videos where the height is not known) with a bitrate of at least 500 KBit/s. You can also use the filters with `all` to download all formats that satisfy the filter, e.g. `-f "all[vcodec=none]"` selects all audio-only formats. Formats for which the value is not known are excluded unless you put a question mark (`?`) after the operator. You can combine format filters, so `-f "[height<=?720][tbr>500]"` selects up to 720p videos (or videos where the height is not known) with a bitrate of at least 500 KBit/s. You can also use the filters with `all` to download all formats that satisfy the filter, e.g. `-f "all[vcodec=none]"` selects all audio-only formats.
@@ -1487,9 +1511,9 @@ The available fields are:
- `source`: The preference of the source - `source`: The preference of the source
- `proto`: Protocol used for download (`https`/`ftps` > `http`/`ftp` > `m3u8_native`/`m3u8` > `http_dash_segments`> `websocket_frag` > `mms`/`rtsp` > `f4f`/`f4m`) - `proto`: Protocol used for download (`https`/`ftps` > `http`/`ftp` > `m3u8_native`/`m3u8` > `http_dash_segments`> `websocket_frag` > `mms`/`rtsp` > `f4f`/`f4m`)
- `vcodec`: Video Codec (`av01` > `vp9.2` > `vp9` > `h265` > `h264` > `vp8` > `h263` > `theora` > other) - `vcodec`: Video Codec (`av01` > `vp9.2` > `vp9` > `h265` > `h264` > `vp8` > `h263` > `theora` > other)
- `acodec`: Audio Codec (`flac`/`alac` > `wav`/`aiff` > `opus` > `vorbis` > `aac` > `mp4a` > `mp3` > `eac3` > `ac3` > `dts` > other) - `acodec`: Audio Codec (`flac`/`alac` > `wav`/`aiff` > `opus` > `vorbis` > `aac` > `mp4a` > `mp3` > `ac4` > `eac3` > `ac3` > `dts` > other)
- `codec`: Equivalent to `vcodec,acodec` - `codec`: Equivalent to `vcodec,acodec`
- `vext`: Video Extension (`mp4` > `webm` > `flv` > other). If `--prefer-free-formats` is used, `webm` is preferred. - `vext`: Video Extension (`mp4` > `mov` > `webm` > `flv` > other). If `--prefer-free-formats` is used, `webm` is preferred.
- `aext`: Audio Extension (`m4a` > `aac` > `mp3` > `ogg` > `opus` > `webm` > other). If `--prefer-free-formats` is used, the order changes to `ogg` > `opus` > `webm` > `mp3` > `m4a` > `aac` - `aext`: Audio Extension (`m4a` > `aac` > `mp3` > `ogg` > `opus` > `webm` > other). If `--prefer-free-formats` is used, the order changes to `ogg` > `opus` > `webm` > `mp3` > `m4a` > `aac`
- `ext`: Equivalent to `vext,aext` - `ext`: Equivalent to `vext,aext`
- `filesize`: Exact filesize, if known in advance - `filesize`: Exact filesize, if known in advance
@@ -1565,7 +1589,7 @@ $ yt-dlp -S "+size,+br"
$ yt-dlp -f "bv*[ext=mp4]+ba[ext=m4a]/b[ext=mp4] / bv*+ba/b" $ yt-dlp -f "bv*[ext=mp4]+ba[ext=m4a]/b[ext=mp4] / bv*+ba/b"
# Download the best video with the best extension # Download the best video with the best extension
# (For video, mp4 > webm > flv. For audio, m4a > aac > mp3 ...) # (For video, mp4 > mov > webm > flv. For audio, m4a > aac > mp3 ...)
$ yt-dlp -S "ext" $ yt-dlp -S "ext"
@@ -1717,10 +1741,12 @@ $ yt-dlp --replace-in-metadata "title,uploader" "[ _]" "-"
Some extractors accept additional arguments which can be passed using `--extractor-args KEY:ARGS`. `ARGS` is a `;` (semicolon) separated string of `ARG=VAL1,VAL2`. E.g. `--extractor-args "youtube:player-client=android_embedded,web;include_live_dash" --extractor-args "funimation:version=uncut"` Some extractors accept additional arguments which can be passed using `--extractor-args KEY:ARGS`. `ARGS` is a `;` (semicolon) separated string of `ARG=VAL1,VAL2`. E.g. `--extractor-args "youtube:player-client=android_embedded,web;include_live_dash" --extractor-args "funimation:version=uncut"`
Note: In CLI, `ARG` can use `-` instead of `_`; e.g. `youtube:player-client"` becomes `youtube:player_client"`
The following extractors use this feature: The following extractors use this feature:
#### youtube #### youtube
* `lang`: Language code to prefer translated metadata of this language (case-sensitive). By default, the video primary language metadata is preferred, with a fallback to `en` translated. See [youtube.py](https://github.com/yt-dlp/yt-dlp/blob/c26f9b991a0681fd3ea548d535919cec1fbbd430/yt_dlp/extractor/youtube.py#L381-L390) for list of supported content language codes * `lang`: Prefer translated metadata (`title`, `description` etc) of this language code (case-sensitive). By default, the video primary language metadata is preferred, with a fallback to `en` translated. See [youtube.py](https://github.com/yt-dlp/yt-dlp/blob/c26f9b991a0681fd3ea548d535919cec1fbbd430/yt_dlp/extractor/youtube.py#L381-L390) for list of supported content language codes
* `skip`: One or more of `hls`, `dash` or `translated_subs` to skip extraction of the m3u8 manifests, dash manifests and [auto-translated subtitles](https://github.com/yt-dlp/yt-dlp/issues/4090#issuecomment-1158102032) respectively * `skip`: One or more of `hls`, `dash` or `translated_subs` to skip extraction of the m3u8 manifests, dash manifests and [auto-translated subtitles](https://github.com/yt-dlp/yt-dlp/issues/4090#issuecomment-1158102032) respectively
* `player_client`: Clients to extract video data from. The main clients are `web`, `android` and `ios` with variants `_music`, `_embedded`, `_embedscreen`, `_creator` (e.g. `web_embedded`); and `mweb` and `tv_embedded` (agegate bypass) with no variants. By default, `android,web` is used, but `tv_embedded` and `creator` variants are added as required for age-gated videos. Similarly, the music variants are added for `music.youtube.com` urls. You can use `all` to use all the clients, and `default` for the default clients. * `player_client`: Clients to extract video data from. The main clients are `web`, `android` and `ios` with variants `_music`, `_embedded`, `_embedscreen`, `_creator` (e.g. `web_embedded`); and `mweb` and `tv_embedded` (agegate bypass) with no variants. By default, `android,web` is used, but `tv_embedded` and `creator` variants are added as required for age-gated videos. Similarly, the music variants are added for `music.youtube.com` urls. You can use `all` to use all the clients, and `default` for the default clients.
* `player_skip`: Skip some network requests that are generally needed for robust extraction. One or more of `configs` (skip client configs), `webpage` (skip initial webpage), `js` (skip js player). While these options can help reduce the number of requests needed or avoid some rate-limiting, they could cause some issues. See [#860](https://github.com/yt-dlp/yt-dlp/pull/860) for more details * `player_skip`: Skip some network requests that are generally needed for robust extraction. One or more of `configs` (skip client configs), `webpage` (skip initial webpage), `js` (skip js player). While these options can help reduce the number of requests needed or avoid some rate-limiting, they could cause some issues. See [#860](https://github.com/yt-dlp/yt-dlp/pull/860) for more details
@@ -1735,6 +1761,9 @@ The following extractors use this feature:
* `skip`: One or more of `webpage` (skip initial webpage download), `authcheck` (allow the download of playlists requiring authentication when no initial webpage is downloaded. This may cause unwanted behavior, see [#1122](https://github.com/yt-dlp/yt-dlp/pull/1122) for more details) * `skip`: One or more of `webpage` (skip initial webpage download), `authcheck` (allow the download of playlists requiring authentication when no initial webpage is downloaded. This may cause unwanted behavior, see [#1122](https://github.com/yt-dlp/yt-dlp/pull/1122) for more details)
* `approximate_date`: Extract approximate `upload_date` and `timestamp` in flat-playlist. This may cause date-based filters to be slightly off * `approximate_date`: Extract approximate `upload_date` and `timestamp` in flat-playlist. This may cause date-based filters to be slightly off
#### generic
* `fragment_query`: Passthrough any query in mpd/m3u8 manifest URLs to their fragments. Does not apply to ffmpeg
#### funimation #### funimation
* `language`: Audio languages to extract, e.g. `funimation:language=english,japanese` * `language`: Audio languages to extract, e.g. `funimation:language=english,japanese`
* `version`: The video version to extract - `uncut` or `simulcast` * `version`: The video version to extract - `uncut` or `simulcast`
@@ -1761,6 +1790,7 @@ The following extractors use this feature:
* `dr`: dynamic range to ignore - one or more of `sdr`, `hdr10`, `dv` * `dr`: dynamic range to ignore - one or more of `sdr`, `hdr10`, `dv`
#### tiktok #### tiktok
* `api_hostname`: Hostname to use for mobile API requests, e.g. `api-h2.tiktokv.com`
* `app_version`: App version to call mobile APIs with - should be set along with `manifest_app_version`, e.g. `20.2.1` * `app_version`: App version to call mobile APIs with - should be set along with `manifest_app_version`, e.g. `20.2.1`
* `manifest_app_version`: Numeric app version to call mobile APIs with, e.g. `221` * `manifest_app_version`: Numeric app version to call mobile APIs with, e.g. `221`
@@ -1770,26 +1800,78 @@ The following extractors use this feature:
#### twitter #### twitter
* `force_graphql`: Force usage of the GraphQL API. By default it will only be used if login cookies are provided * `force_graphql`: Force usage of the GraphQL API. By default it will only be used if login cookies are provided
NOTE: These options may be changed/removed in the future without concern for backward compatibility **Note**: These options may be changed/removed in the future without concern for backward compatibility
<!-- MANPAGE: MOVE "INSTALLATION" SECTION HERE --> <!-- MANPAGE: MOVE "INSTALLATION" SECTION HERE -->
# PLUGINS # PLUGINS
Plugins are loaded from `<root-dir>/ytdlp_plugins/<type>/__init__.py`; where `<root-dir>` is the directory of the binary (`<root-dir>/yt-dlp`), or the root directory of the module if you are running directly from source-code (`<root dir>/yt_dlp/__main__.py`). Plugins are currently not supported for the `pip` version Note that **all** plugins are imported even if not invoked, and that **there are no checks** performed on plugin code. **Use plugins at your own risk and only if you trust the code!**
Plugins can be of `<type>`s `extractor` or `postprocessor`. Extractor plugins do not need to be enabled from the CLI and are automatically invoked when the input URL is suitable for it. Postprocessor plugins can be invoked using `--use-postprocessor NAME`. Plugins can be of `<type>`s `extractor` or `postprocessor`.
- Extractor plugins do not need to be enabled from the CLI and are automatically invoked when the input URL is suitable for it.
- Extractor plugins take priority over builtin extractors.
- Postprocessor plugins can be invoked using `--use-postprocessor NAME`.
See [ytdlp_plugins](ytdlp_plugins) for example plugins.
Note that **all** plugins are imported even if not invoked, and that **there are no checks** performed on plugin code. Use plugins at your own risk and only if you trust the code Plugins are loaded from the namespace packages `yt_dlp_plugins.extractor` and `yt_dlp_plugins.postprocessor`.
If you are a plugin author, add [ytdlp-plugins](https://github.com/topics/ytdlp-plugins) as a topic to your repository for discoverability In other words, the file structure on the disk looks something like:
yt_dlp_plugins/
extractor/
myplugin.py
postprocessor/
myplugin.py
yt-dlp looks for these `yt_dlp_plugins` namespace folders in many locations (see below) and loads in plugins from **all** of them.
See the [wiki for some known plugins](https://github.com/yt-dlp/yt-dlp/wiki/Plugins) See the [wiki for some known plugins](https://github.com/yt-dlp/yt-dlp/wiki/Plugins)
## Installing Plugins
Plugins can be installed using various methods and locations.
1. **Configuration directories**:
Plugin packages (containing a `yt_dlp_plugins` namespace folder) can be dropped into the following standard [configuration locations](#configuration):
* **User Plugins**
* `${XDG_CONFIG_HOME}/yt-dlp/plugins/<package name>/yt_dlp_plugins/` (recommended on Linux/macOS)
* `${XDG_CONFIG_HOME}/yt-dlp-plugins/<package name>/yt_dlp_plugins/`
* `${APPDATA}/yt-dlp/plugins/<package name>/yt_dlp_plugins/` (recommended on Windows)
* `${APPDATA}/yt-dlp-plugins/<package name>/yt_dlp_plugins/`
* `~/.yt-dlp/plugins/<package name>/yt_dlp_plugins/`
* `~/yt-dlp-plugins/<package name>/yt_dlp_plugins/`
* **System Plugins**
* `/etc/yt-dlp/plugins/<package name>/yt_dlp_plugins/`
* `/etc/yt-dlp-plugins/<package name>/yt_dlp_plugins/`
2. **Executable location**: Plugin packages can similarly be installed in a `yt-dlp-plugins` directory under the executable location:
* Binary: where `<root-dir>/yt-dlp.exe`, `<root-dir>/yt-dlp-plugins/<package name>/yt_dlp_plugins/`
* Source: where `<root-dir>/yt_dlp/__main__.py`, `<root-dir>/yt-dlp-plugins/<package name>/yt_dlp_plugins/`
3. **pip and other locations in `PYTHONPATH`**
* Plugin packages can be installed and managed using `pip`. See [yt-dlp-sample-plugins](https://github.com/yt-dlp/yt-dlp-sample-plugins) for an example.
* Note: plugin files between plugin packages installed with pip must have unique filenames.
* Any path in `PYTHONPATH` is searched in for the `yt_dlp_plugins` namespace folder.
* Note: This does not apply for Pyinstaller/py2exe builds.
`.zip`, `.egg` and `.whl` archives containing a `yt_dlp_plugins` namespace folder in their root are also supported as plugin packages.
* e.g. `${XDG_CONFIG_HOME}/yt-dlp/plugins/mypluginpkg.zip` where `mypluginpkg.zip` contains `yt_dlp_plugins/<type>/myplugin.py`
Run yt-dlp with `--verbose` to check if the plugin has been loaded.
## Developing Plugins
See the [yt-dlp-sample-plugins](https://github.com/yt-dlp/yt-dlp-sample-plugins) repo for a template plugin package and the [Plugin Development](https://github.com/yt-dlp/yt-dlp/wiki/Plugin-Development) section of the wiki for a plugin development guide.
All public classes with a name ending in `IE`/`PP` are imported from each file for extractors and postprocessors repectively. This respects underscore prefix (e.g. `_MyBasePluginIE` is private) and `__all__`. Modules can similarly be excluded by prefixing the module name with an underscore (e.g. `_myplugin.py`).
To replace an existing extractor with a subclass of one, set the `plugin_name` class keyword argument (e.g. `class MyPluginIE(ABuiltInIE, plugin_name='myplugin')` will replace `ABuiltInIE` with `MyPluginIE`). Since the extractor replaces the parent, you should exclude the subclass extractor from being imported separately by making it private using one of the methods described above.
If you are a plugin author, add [yt-dlp-plugins](https://github.com/topics/yt-dlp-plugins) as a topic to your repository for discoverability.
See the [Developer Instructions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions) on how to write and test an extractor.
# EMBEDDING YT-DLP # EMBEDDING YT-DLP
@@ -1807,7 +1889,7 @@ with YoutubeDL() as ydl:
ydl.download(URLS) ydl.download(URLS)
``` ```
Most likely, you'll want to use various options. For a list of options available, have a look at [`yt_dlp/YoutubeDL.py`](yt_dlp/YoutubeDL.py#L180). Most likely, you'll want to use various options. For a list of options available, have a look at [`yt_dlp/YoutubeDL.py`](yt_dlp/YoutubeDL.py#L184).
**Tip**: If you are porting your code from youtube-dl to yt-dlp, one important point to look out for is that we do not guarantee the return value of `YoutubeDL.extract_info` to be json serializable, or even be a dictionary. It will be dictionary-like, but if you want to ensure it is a serializable dictionary, pass it through `YoutubeDL.sanitize_info` as shown in the [example below](#extracting-information) **Tip**: If you are porting your code from youtube-dl to yt-dlp, one important point to look out for is that we do not guarantee the return value of `YoutubeDL.extract_info` to be json serializable, or even be a dictionary. It will be dictionary-like, but if you want to ensure it is a serializable dictionary, pass it through `YoutubeDL.sanitize_info` as shown in the [example below](#extracting-information)
+1 -1
View File
@@ -10,7 +10,7 @@ from ..utils import (
) )
# These bloat the lazy_extractors, so allow them to passthrough silently # These bloat the lazy_extractors, so allow them to passthrough silently
ALLOWED_CLASSMETHODS = {'get_testcases', 'extract_from_webpage'} ALLOWED_CLASSMETHODS = {'extract_from_webpage', 'get_testcases', 'get_webpage_testcases'}
_WARNED = False _WARNED = False
+1 -1
View File
@@ -58,7 +58,7 @@ NO_SKIP = '''
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
description: Fill all fields even if you think it is irrelevant for the issue description: Fill all fields even if you think it is irrelevant for the issue
options: options:
- label: I understand that I will be **blocked** if I remove or skip any mandatory\\* field - label: I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\\* field
required: true required: true
'''.strip() '''.strip()
+13 -2
View File
@@ -14,10 +14,17 @@ from devscripts.utils import get_filename_args, read_file, write_file
NO_ATTR = object() NO_ATTR = object()
STATIC_CLASS_PROPERTIES = [ STATIC_CLASS_PROPERTIES = [
'IE_NAME', 'IE_DESC', 'SEARCH_KEY', '_VALID_URL', '_WORKING', '_ENABLED', '_NETRC_MACHINE', 'age_limit' 'IE_NAME', '_ENABLED', '_VALID_URL', # Used for URL matching
'_WORKING', 'IE_DESC', '_NETRC_MACHINE', 'SEARCH_KEY', # Used for --extractor-descriptions
'age_limit', # Used for --age-limit (evaluated)
'_RETURN_TYPE', # Accessed in CLI only with instance (evaluated)
] ]
CLASS_METHODS = [ CLASS_METHODS = [
'ie_key', 'working', 'description', 'suitable', '_match_valid_url', '_match_id', 'get_temp_id', 'is_suitable' 'ie_key', 'suitable', '_match_valid_url', # Used for URL matching
'working', 'get_temp_id', '_match_id', # Accessed just before instance creation
'description', # Used for --extractor-descriptions
'is_suitable', # Used for --age-limit
'supports_login', 'is_single_video', # Accessed in CLI only with instance
] ]
IE_TEMPLATE = ''' IE_TEMPLATE = '''
class {name}({bases}): class {name}({bases}):
@@ -33,8 +40,12 @@ def main():
_ALL_CLASSES = get_all_ies() # Must be before import _ALL_CLASSES = get_all_ies() # Must be before import
import yt_dlp.plugins
from yt_dlp.extractor.common import InfoExtractor, SearchInfoExtractor from yt_dlp.extractor.common import InfoExtractor, SearchInfoExtractor
# Filter out plugins
_ALL_CLASSES = [cls for cls in _ALL_CLASSES if not cls.__module__.startswith(f'{yt_dlp.plugins.PACKAGE_NAME}.')]
DummyInfoExtractor = type('InfoExtractor', (InfoExtractor,), {'IE_NAME': NO_ATTR}) DummyInfoExtractor = type('InfoExtractor', (InfoExtractor,), {'IE_NAME': NO_ATTR})
module_src = '\n'.join(( module_src = '\n'.join((
MODULE_TEMPLATE, MODULE_TEMPLATE,
+6 -26
View File
@@ -37,7 +37,7 @@ def main():
'--icon=devscripts/logo.ico', '--icon=devscripts/logo.ico',
'--upx-exclude=vcruntime140.dll', '--upx-exclude=vcruntime140.dll',
'--noconfirm', '--noconfirm',
*dependency_options(), '--additional-hooks-dir=yt_dlp/__pyinstaller',
*opts, *opts,
'yt_dlp/__main__.py', 'yt_dlp/__main__.py',
] ]
@@ -77,30 +77,6 @@ def version_to_list(version):
return list(map(int, version_list)) + [0] * (4 - len(version_list)) return list(map(int, version_list)) + [0] * (4 - len(version_list))
def dependency_options():
# Due to the current implementation, these are auto-detected, but explicitly add them just in case
dependencies = [pycryptodome_module(), 'mutagen', 'brotli', 'certifi', 'websockets']
excluded_modules = ('youtube_dl', 'youtube_dlc', 'test', 'ytdlp_plugins', 'devscripts')
yield from (f'--hidden-import={module}' for module in dependencies)
yield '--collect-submodules=websockets'
yield from (f'--exclude-module={module}' for module in excluded_modules)
def pycryptodome_module():
try:
import Cryptodome # noqa: F401
except ImportError:
try:
import Crypto # noqa: F401
print('WARNING: Using Crypto since Cryptodome is not available. '
'Install with: pip install pycryptodomex', file=sys.stderr)
return 'Crypto'
except ImportError:
pass
return 'Cryptodome'
def set_version_info(exe, version): def set_version_info(exe, version):
if OS_NAME == 'win32': if OS_NAME == 'win32':
windows_set_version(exe, version) windows_set_version(exe, version)
@@ -109,7 +85,6 @@ def set_version_info(exe, version):
def windows_set_version(exe, version): def windows_set_version(exe, version):
from PyInstaller.utils.win32.versioninfo import ( from PyInstaller.utils.win32.versioninfo import (
FixedFileInfo, FixedFileInfo,
SetVersion,
StringFileInfo, StringFileInfo,
StringStruct, StringStruct,
StringTable, StringTable,
@@ -118,6 +93,11 @@ def windows_set_version(exe, version):
VSVersionInfo, VSVersionInfo,
) )
try:
from PyInstaller.utils.win32.versioninfo import SetVersion
except ImportError: # Pyinstaller >= 5.8
from PyInstaller.utils.win32.versioninfo import write_version_info_to_executable as SetVersion
version_list = version_to_list(version) version_list = version_to_list(version)
suffix = MACHINE and f'_{MACHINE}' suffix = MACHINE and f'_{MACHINE}'
SetVersion(exe, VSVersionInfo( SetVersion(exe, VSVersionInfo(
+5
View File
@@ -0,0 +1,5 @@
[build-system]
build-backend = 'setuptools.build_meta'
# https://github.com/yt-dlp/yt-dlp/issues/5941
# https://github.com/pypa/distutils/issues/17
requires = ['setuptools > 50']
+1 -1
View File
@@ -26,7 +26,7 @@ markers =
[tox:tox] [tox:tox]
skipsdist = true skipsdist = true
envlist = py{36,37,38,39,310},pypy{36,37,38,39} envlist = py{36,37,38,39,310,311},pypy{36,37,38,39}
skip_missing_interpreters = true skip_missing_interpreters = true
[testenv] # tox [testenv] # tox
+10 -3
View File
@@ -1,8 +1,12 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
import os.path # Allow execution from anywhere
import subprocess import os
import sys import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import subprocess
import warnings import warnings
try: try:
@@ -88,7 +92,10 @@ def build_params():
params = {'data_files': data_files} params = {'data_files': data_files}
if setuptools_available: if setuptools_available:
params['entry_points'] = {'console_scripts': ['yt-dlp = yt_dlp:main']} params['entry_points'] = {
'console_scripts': ['yt-dlp = yt_dlp:main'],
'pyinstaller40': ['hook-dirs = yt_dlp.__pyinstaller:get_hook_dirs'],
}
else: else:
params['scripts'] = ['yt-dlp'] params['scripts'] = ['yt-dlp']
return params return params
+69 -11
View File
@@ -51,6 +51,8 @@
- **afreecatv:live**: [<abbr title="netrc machine"><em>afreecatv</em></abbr>] afreecatv.com - **afreecatv:live**: [<abbr title="netrc machine"><em>afreecatv</em></abbr>] afreecatv.com
- **afreecatv:user** - **afreecatv:user**
- **AirMozilla** - **AirMozilla**
- **AirTV**
- **AitubeKZVideo**
- **AliExpressLive** - **AliExpressLive**
- **AlJazeera** - **AlJazeera**
- **Allocine** - **Allocine**
@@ -60,11 +62,16 @@
- **Alura**: [<abbr title="netrc machine"><em>alura</em></abbr>] - **Alura**: [<abbr title="netrc machine"><em>alura</em></abbr>]
- **AluraCourse**: [<abbr title="netrc machine"><em>aluracourse</em></abbr>] - **AluraCourse**: [<abbr title="netrc machine"><em>aluracourse</em></abbr>]
- **Amara** - **Amara**
- **AmazonMiniTV**
- **amazonminitv:season**: Amazon MiniTV Season, "minitv:season:" prefix
- **amazonminitv:series**: Amazon MiniTV Series, "minitv:series:" prefix
- **AmazonReviews**
- **AmazonStore** - **AmazonStore**
- **AMCNetworks** - **AMCNetworks**
- **AmericasTestKitchen** - **AmericasTestKitchen**
- **AmericasTestKitchenSeason** - **AmericasTestKitchenSeason**
- **AmHistoryChannel** - **AmHistoryChannel**
- **AnchorFMEpisode**
- **anderetijden**: npo.nl, ntr.nl, omroepwnl.nl, zapp.nl and npo3.nl - **anderetijden**: npo.nl, ntr.nl, omroepwnl.nl, zapp.nl and npo3.nl
- **Angel** - **Angel**
- **AnimalPlanet** - **AnimalPlanet**
@@ -130,6 +137,8 @@
- **BBVTV**: [<abbr title="netrc machine"><em>bbvtv</em></abbr>] - **BBVTV**: [<abbr title="netrc machine"><em>bbvtv</em></abbr>]
- **BBVTVLive**: [<abbr title="netrc machine"><em>bbvtv</em></abbr>] - **BBVTVLive**: [<abbr title="netrc machine"><em>bbvtv</em></abbr>]
- **BBVTVRecordings**: [<abbr title="netrc machine"><em>bbvtv</em></abbr>] - **BBVTVRecordings**: [<abbr title="netrc machine"><em>bbvtv</em></abbr>]
- **BeatBumpPlaylist**
- **BeatBumpVideo**
- **Beatport** - **Beatport**
- **Beeg** - **Beeg**
- **BehindKink** - **BehindKink**
@@ -157,7 +166,7 @@
- **BilibiliSpacePlaylist** - **BilibiliSpacePlaylist**
- **BilibiliSpaceVideo** - **BilibiliSpaceVideo**
- **BiliIntl**: [<abbr title="netrc machine"><em>biliintl</em></abbr>] - **BiliIntl**: [<abbr title="netrc machine"><em>biliintl</em></abbr>]
- **BiliIntlSeries**: [<abbr title="netrc machine"><em>biliintl</em></abbr>] - **biliIntl:series**: [<abbr title="netrc machine"><em>biliintl</em></abbr>]
- **BiliLive** - **BiliLive**
- **BioBioChileTV** - **BioBioChileTV**
- **Biography** - **Biography**
@@ -169,6 +178,7 @@
- **BlackboardCollaborate** - **BlackboardCollaborate**
- **BleacherReport** - **BleacherReport**
- **BleacherReportCMS** - **BleacherReportCMS**
- **blerp**
- **blogger.com** - **blogger.com**
- **Bloomberg** - **Bloomberg**
- **BokeCC** - **BokeCC**
@@ -176,6 +186,7 @@
- **BooyahClips** - **BooyahClips**
- **BostonGlobe** - **BostonGlobe**
- **Box** - **Box**
- **BoxCastVideo**
- **Bpb**: Bundeszentrale für politische Bildung - **Bpb**: Bundeszentrale für politische Bildung
- **BR**: Bayerischer Rundfunk - **BR**: Bayerischer Rundfunk
- **BravoTV** - **BravoTV**
@@ -345,6 +356,8 @@
- **DrTuber** - **DrTuber**
- **drtv** - **drtv**
- **drtv:live** - **drtv:live**
- **drtv:season**
- **drtv:series**
- **DTube** - **DTube**
- **duboku**: www.duboku.io - **duboku**: www.duboku.io
- **duboku:list**: www.duboku.io entire series - **duboku:list**: www.duboku.io entire series
@@ -354,6 +367,7 @@
- **dw:article** - **dw:article**
- **EaglePlatform** - **EaglePlatform**
- **EbaumsWorld** - **EbaumsWorld**
- **Ebay**
- **EchoMsk** - **EchoMsk**
- **egghead:course**: egghead.io course - **egghead:course**: egghead.io course
- **egghead:lesson**: egghead.io lesson - **egghead:lesson**: egghead.io lesson
@@ -387,6 +401,7 @@
- **ESPNCricInfo** - **ESPNCricInfo**
- **EsriVideo** - **EsriVideo**
- **Europa** - **Europa**
- **EuroParlWebstream**
- **EuropeanTour** - **EuropeanTour**
- **Eurosport** - **Eurosport**
- **EUScreen** - **EUScreen**
@@ -584,6 +599,7 @@
- **ivi**: ivi.ru - **ivi**: ivi.ru
- **ivi:compilation**: ivi.ru compilations - **ivi:compilation**: ivi.ru compilations
- **ivideon**: Ivideon TV - **ivideon**: Ivideon TV
- **IVXPlayer**
- **Iwara** - **Iwara**
- **iwara:playlist** - **iwara:playlist**
- **iwara:user** - **iwara:user**
@@ -599,6 +615,8 @@
- **JWPlatform** - **JWPlatform**
- **Kakao** - **Kakao**
- **Kaltura** - **Kaltura**
- **Kanal2**
- **KankaNews**
- **Karaoketv** - **Karaoketv**
- **KarriereVideos** - **KarriereVideos**
- **Katsomo** - **Katsomo**
@@ -607,10 +625,13 @@
- **Ketnet** - **Ketnet**
- **khanacademy** - **khanacademy**
- **khanacademy:unit** - **khanacademy:unit**
- **Kick**
- **Kicker** - **Kicker**
- **KickStarter** - **KickStarter**
- **KickVOD**
- **KinjaEmbed** - **KinjaEmbed**
- **KinoPoisk** - **KinoPoisk**
- **Kommunetv**
- **KompasVideo** - **KompasVideo**
- **KonserthusetPlay** - **KonserthusetPlay**
- **Koo** - **Koo**
@@ -709,6 +730,7 @@
- **Mediasite** - **Mediasite**
- **MediasiteCatalog** - **MediasiteCatalog**
- **MediasiteNamedCatalog** - **MediasiteNamedCatalog**
- **MediaStream**
- **MediaWorksNZVOD** - **MediaWorksNZVOD**
- **Medici** - **Medici**
- **megaphone.fm**: megaphone.fm embedded players - **megaphone.fm**: megaphone.fm embedded players
@@ -757,6 +779,7 @@
- **Mofosex** - **Mofosex**
- **MofosexEmbed** - **MofosexEmbed**
- **Mojvideo** - **Mojvideo**
- **MonsterSirenHypergryphMusic**
- **Morningstar**: morningstar.com - **Morningstar**: morningstar.com
- **Motherless** - **Motherless**
- **MotherlessGroup** - **MotherlessGroup**
@@ -845,6 +868,7 @@
- **NetPlusTVRecordings**: [<abbr title="netrc machine"><em>netplus</em></abbr>] - **NetPlusTVRecordings**: [<abbr title="netrc machine"><em>netplus</em></abbr>]
- **Netverse** - **Netverse**
- **NetversePlaylist** - **NetversePlaylist**
- **NetverseSearch**: "netsearch:" prefix
- **Netzkino** - **Netzkino**
- **Newgrounds** - **Newgrounds**
- **Newgrounds:playlist** - **Newgrounds:playlist**
@@ -861,6 +885,8 @@
- **NFHSNetwork** - **NFHSNetwork**
- **nfl.com** - **nfl.com**
- **nfl.com:article** - **nfl.com:article**
- **nfl.com:plus:episode**
- **nfl.com:plus:replay**
- **NhkForSchoolBangumi** - **NhkForSchoolBangumi**
- **NhkForSchoolProgramList** - **NhkForSchoolProgramList**
- **NhkForSchoolSubject**: Portal page for each school subjects, like Japanese (kokugo, 国語) or math (sansuu/suugaku or 算数・数学) - **NhkForSchoolSubject**: Portal page for each school subjects, like Japanese (kokugo, 国語) or math (sansuu/suugaku or 算数・数学)
@@ -873,7 +899,7 @@
- **nickelodeonru** - **nickelodeonru**
- **nicknight** - **nicknight**
- **niconico**: [<abbr title="netrc machine"><em>niconico</em></abbr>] ニコニコ動画 - **niconico**: [<abbr title="netrc machine"><em>niconico</em></abbr>] ニコニコ動画
- **niconico:history**: NicoNico user history. Requires cookies. - **niconico:history**: NicoNico user history or likes. Requires cookies.
- **niconico:playlist** - **niconico:playlist**
- **niconico:series** - **niconico:series**
- **niconico:tag**: NicoNico video tag URLs - **niconico:tag**: NicoNico video tag URLs
@@ -887,6 +913,7 @@
- **njoy:embed** - **njoy:embed**
- **NJPWWorld**: [<abbr title="netrc machine"><em>njpwworld</em></abbr>] 新日本プロレスワールド - **NJPWWorld**: [<abbr title="netrc machine"><em>njpwworld</em></abbr>] 新日本プロレスワールド
- **NobelPrize** - **NobelPrize**
- **NoicePodcast**
- **NonkTube** - **NonkTube**
- **NoodleMagazine** - **NoodleMagazine**
- **Noovo** - **Noovo**
@@ -922,6 +949,7 @@
- **NYTimesArticle** - **NYTimesArticle**
- **NYTimesCooking** - **NYTimesCooking**
- **nzherald** - **nzherald**
- **NZOnScreen**
- **NZZ** - **NZZ**
- **ocw.mit.edu** - **ocw.mit.edu**
- **OdaTV** - **OdaTV**
@@ -931,8 +959,10 @@
- **OktoberfestTV** - **OktoberfestTV**
- **OlympicsReplay** - **OlympicsReplay**
- **on24**: ON24 - **on24**: ON24
- **OnDemandChinaEpisode**
- **OnDemandKorea** - **OnDemandKorea**
- **OneFootball** - **OneFootball**
- **OnePlacePodcast**
- **onet.pl** - **onet.pl**
- **onet.tv** - **onet.tv**
- **onet.tv:channel** - **onet.tv:channel**
@@ -1022,11 +1052,13 @@
- **PokerGoCollection**: [<abbr title="netrc machine"><em>pokergo</em></abbr>] - **PokerGoCollection**: [<abbr title="netrc machine"><em>pokergo</em></abbr>]
- **PolsatGo** - **PolsatGo**
- **PolskieRadio** - **PolskieRadio**
- **polskieradio:audition**
- **polskieradio:category**
- **polskieradio:kierowcow** - **polskieradio:kierowcow**
- **polskieradio:legacy**
- **polskieradio:player** - **polskieradio:player**
- **polskieradio:podcast** - **polskieradio:podcast**
- **polskieradio:podcast:list** - **polskieradio:podcast:list**
- **PolskieRadioCategory**
- **Popcorntimes** - **Popcorntimes**
- **PopcornTV** - **PopcornTV**
- **PornCom** - **PornCom**
@@ -1042,7 +1074,10 @@
- **Pornotube** - **Pornotube**
- **PornoVoisines** - **PornoVoisines**
- **PornoXO** - **PornoXO**
- **PornTop**
- **PornTube** - **PornTube**
- **Pr0gramm**
- **Pr0grammStatic**
- **PrankCast** - **PrankCast**
- **PremiershipRugby** - **PremiershipRugby**
- **PressTV** - **PressTV**
@@ -1094,6 +1129,8 @@
- **RaiSudtirol** - **RaiSudtirol**
- **RayWenderlich** - **RayWenderlich**
- **RayWenderlichCourse** - **RayWenderlichCourse**
- **RbgTum**
- **RbgTumCourse**
- **RBMARadio** - **RBMARadio**
- **RCS** - **RCS**
- **RCSEmbeds** - **RCSEmbeds**
@@ -1128,6 +1165,7 @@
- **RoosterTeethSeries**: [<abbr title="netrc machine"><em>roosterteeth</em></abbr>] - **RoosterTeethSeries**: [<abbr title="netrc machine"><em>roosterteeth</em></abbr>]
- **RottenTomatoes** - **RottenTomatoes**
- **Rozhlas** - **Rozhlas**
- **RozhlasVltava**
- **RTBF**: [<abbr title="netrc machine"><em>rtbf</em></abbr>] - **RTBF**: [<abbr title="netrc machine"><em>rtbf</em></abbr>]
- **RTDocumentry** - **RTDocumentry**
- **RTDocumentryPlaylist** - **RTDocumentryPlaylist**
@@ -1155,6 +1193,7 @@
- **rtvslo.si** - **rtvslo.si**
- **RUHD** - **RUHD**
- **Rule34Video** - **Rule34Video**
- **Rumble**
- **RumbleChannel** - **RumbleChannel**
- **RumbleEmbed** - **RumbleEmbed**
- **Ruptly** - **Ruptly**
@@ -1189,6 +1228,7 @@
- **screen.yahoo:search**: Yahoo screen search; "yvsearch:" prefix - **screen.yahoo:search**: Yahoo screen search; "yvsearch:" prefix
- **Screen9** - **Screen9**
- **Screencast** - **Screencast**
- **Screencastify**
- **ScreencastOMatic** - **ScreencastOMatic**
- **ScrippsNetworks** - **ScrippsNetworks**
- **scrippsnetworks:watch** - **scrippsnetworks:watch**
@@ -1212,6 +1252,7 @@
- **ShugiinItvLive**: 衆議院インターネット審議中継 - **ShugiinItvLive**: 衆議院インターネット審議中継
- **ShugiinItvLiveRoom**: 衆議院インターネット審議中継 (中継) - **ShugiinItvLiveRoom**: 衆議院インターネット審議中継 (中継)
- **ShugiinItvVod**: 衆議院インターネット審議中継 (ビデオライブラリ) - **ShugiinItvVod**: 衆議院インターネット審議中継 (ビデオライブラリ)
- **SibnetEmbed**
- **simplecast** - **simplecast**
- **simplecast:episode** - **simplecast:episode**
- **simplecast:podcast** - **simplecast:podcast**
@@ -1227,7 +1268,7 @@
- **skynewsarabia:video** - **skynewsarabia:video**
- **SkyNewsAU** - **SkyNewsAU**
- **Slideshare** - **Slideshare**
- **SlidesLive**: (**Currently broken**) - **SlidesLive**
- **Slutload** - **Slutload**
- **Smotrim** - **Smotrim**
- **Snotr** - **Snotr**
@@ -1241,6 +1282,7 @@
- **soundcloud:set**: [<abbr title="netrc machine"><em>soundcloud</em></abbr>] - **soundcloud:set**: [<abbr title="netrc machine"><em>soundcloud</em></abbr>]
- **soundcloud:trackstation**: [<abbr title="netrc machine"><em>soundcloud</em></abbr>] - **soundcloud:trackstation**: [<abbr title="netrc machine"><em>soundcloud</em></abbr>]
- **soundcloud:user**: [<abbr title="netrc machine"><em>soundcloud</em></abbr>] - **soundcloud:user**: [<abbr title="netrc machine"><em>soundcloud</em></abbr>]
- **soundcloud:user:permalink**: [<abbr title="netrc machine"><em>soundcloud</em></abbr>]
- **SoundcloudEmbed** - **SoundcloudEmbed**
- **soundgasm** - **soundgasm**
- **soundgasm:profile** - **soundgasm:profile**
@@ -1352,10 +1394,14 @@
- **ThisAmericanLife** - **ThisAmericanLife**
- **ThisAV** - **ThisAV**
- **ThisOldHouse** - **ThisOldHouse**
- **ThisVid**
- **ThisVidMember**
- **ThisVidPlaylist**
- **ThreeSpeak** - **ThreeSpeak**
- **ThreeSpeakUser** - **ThreeSpeakUser**
- **TikTok** - **TikTok**
- **tiktok:effect**: (**Currently broken**) - **tiktok:effect**: (**Currently broken**)
- **tiktok:live**
- **tiktok:sound**: (**Currently broken**) - **tiktok:sound**: (**Currently broken**)
- **tiktok:tag**: (**Currently broken**) - **tiktok:tag**: (**Currently broken**)
- **tiktok:user**: (**Currently broken**) - **tiktok:user**: (**Currently broken**)
@@ -1383,6 +1429,7 @@
- **TrovoChannelClip**: All Clips of a trovo.live channel; "trovoclip:" prefix - **TrovoChannelClip**: All Clips of a trovo.live channel; "trovoclip:" prefix
- **TrovoChannelVod**: All VODs of a trovo.live channel; "trovovod:" prefix - **TrovoChannelVod**: All VODs of a trovo.live channel; "trovovod:" prefix
- **TrovoVod** - **TrovoVod**
- **TrtCocukVideo**
- **TrueID** - **TrueID**
- **TruNews** - **TruNews**
- **Truth** - **Truth**
@@ -1455,6 +1502,7 @@
- **twitter:card** - **twitter:card**
- **twitter:shortener** - **twitter:shortener**
- **twitter:spaces** - **twitter:spaces**
- **Txxx**
- **udemy**: [<abbr title="netrc machine"><em>udemy</em></abbr>] - **udemy**: [<abbr title="netrc machine"><em>udemy</em></abbr>]
- **udemy:course**: [<abbr title="netrc machine"><em>udemy</em></abbr>] - **udemy:course**: [<abbr title="netrc machine"><em>udemy</em></abbr>]
- **UDNEmbed**: 聯合影音 - **UDNEmbed**: 聯合影音
@@ -1483,6 +1531,7 @@
- **VeeHD** - **VeeHD**
- **Veo** - **Veo**
- **Veoh** - **Veoh**
- **veoh:user**
- **Vesti**: Вести.Ru - **Vesti**: Вести.Ru
- **Vevo** - **Vevo**
- **VevoPlaylist** - **VevoPlaylist**
@@ -1502,6 +1551,11 @@
- **video.sky.it:live** - **video.sky.it:live**
- **VideoDetective** - **VideoDetective**
- **videofy.me** - **videofy.me**
- **VideoKen**
- **VideoKenCategory**
- **VideoKenPlayer**
- **VideoKenPlaylist**
- **VideoKenTopic**
- **videomore** - **videomore**
- **videomore:season** - **videomore:season**
- **videomore:video** - **videomore:video**
@@ -1521,6 +1575,7 @@
- **vimeo:group**: [<abbr title="netrc machine"><em>vimeo</em></abbr>] - **vimeo:group**: [<abbr title="netrc machine"><em>vimeo</em></abbr>]
- **vimeo:likes**: [<abbr title="netrc machine"><em>vimeo</em></abbr>] Vimeo user likes - **vimeo:likes**: [<abbr title="netrc machine"><em>vimeo</em></abbr>] Vimeo user likes
- **vimeo:ondemand**: [<abbr title="netrc machine"><em>vimeo</em></abbr>] - **vimeo:ondemand**: [<abbr title="netrc machine"><em>vimeo</em></abbr>]
- **vimeo:pro**: [<abbr title="netrc machine"><em>vimeo</em></abbr>]
- **vimeo:review**: [<abbr title="netrc machine"><em>vimeo</em></abbr>] Review pages on vimeo - **vimeo:review**: [<abbr title="netrc machine"><em>vimeo</em></abbr>] Review pages on vimeo
- **vimeo:user**: [<abbr title="netrc machine"><em>vimeo</em></abbr>] - **vimeo:user**: [<abbr title="netrc machine"><em>vimeo</em></abbr>]
- **vimeo:watchlater**: [<abbr title="netrc machine"><em>vimeo</em></abbr>] Vimeo watch later list, ":vimeowatchlater" keyword (requires authentication) - **vimeo:watchlater**: [<abbr title="netrc machine"><em>vimeo</em></abbr>] Vimeo watch later list, ":vimeowatchlater" keyword (requires authentication)
@@ -1535,20 +1590,20 @@
- **Viu** - **Viu**
- **viu:ott**: [<abbr title="netrc machine"><em>viu</em></abbr>] - **viu:ott**: [<abbr title="netrc machine"><em>viu</em></abbr>]
- **viu:playlist** - **viu:playlist**
- **ViuOTTIndonesia**
- **Vivo**: vivo.sx - **Vivo**: vivo.sx
- **vk**: [<abbr title="netrc machine"><em>vk</em></abbr>] VK - **vk**: [<abbr title="netrc machine"><em>vk</em></abbr>] VK
- **vk:uservideos**: [<abbr title="netrc machine"><em>vk</em></abbr>] VK - User's Videos - **vk:uservideos**: [<abbr title="netrc machine"><em>vk</em></abbr>] VK - User's Videos
- **vk:wallpost**: [<abbr title="netrc machine"><em>vk</em></abbr>] - **vk:wallpost**: [<abbr title="netrc machine"><em>vk</em></abbr>]
- **vlive**: [<abbr title="netrc machine"><em>vlive</em></abbr>]
- **vlive:channel**: [<abbr title="netrc machine"><em>vlive</em></abbr>]
- **vlive:post**: [<abbr title="netrc machine"><em>vlive</em></abbr>]
- **vm.tiktok** - **vm.tiktok**
- **Vocaroo**
- **Vodlocker** - **Vodlocker**
- **VODPl** - **VODPl**
- **VODPlatform** - **VODPlatform**
- **VoiceRepublic** - **VoiceRepublic**
- **voicy** - **voicy**
- **voicy:channel** - **voicy:channel**
- **VolejTV**
- **Voot** - **Voot**
- **VootSeries** - **VootSeries**
- **VoxMedia** - **VoxMedia**
@@ -1590,7 +1645,9 @@
- **wdr:mobile**: (**Currently broken**) - **wdr:mobile**: (**Currently broken**)
- **WDRElefant** - **WDRElefant**
- **WDRPage** - **WDRPage**
- **web.archive:vlive**: web.archive.org saved vlive videos
- **web.archive:youtube**: web.archive.org saved youtube videos, "ytarchive:" prefix - **web.archive:youtube**: web.archive.org saved youtube videos, "ytarchive:" prefix
- **Webcamerapl**
- **Webcaster** - **Webcaster**
- **WebcasterFeed** - **WebcasterFeed**
- **WebOfStories** - **WebOfStories**
@@ -1604,6 +1661,7 @@
- **wikimedia.org** - **wikimedia.org**
- **Willow** - **Willow**
- **WimTV** - **WimTV**
- **WinSportsVideo**
- **Wistia** - **Wistia**
- **WistiaChannel** - **WistiaChannel**
- **WistiaPlaylist** - **WistiaPlaylist**
@@ -1613,21 +1671,20 @@
- **WorldStarHipHop** - **WorldStarHipHop**
- **wppilot** - **wppilot**
- **wppilot:channels** - **wppilot:channels**
- **WrestleUniversePPV**
- **WrestleUniverseVOD**
- **WSJ**: Wall Street Journal - **WSJ**: Wall Street Journal
- **WSJArticle** - **WSJArticle**
- **WWE** - **WWE**
- **wyborcza:video** - **wyborcza:video**
- **WyborczaPodcast** - **WyborczaPodcast**
- **Xanimu**
- **XBef** - **XBef**
- **XboxClips** - **XboxClips**
- **XFileShare**: XFileShare based sites: Aparat, ClipWatching, GoUnlimited, GoVid, HolaVid, Streamty, TheVideoBee, Uqload, VidBom, vidlo, VidLocker, VidShare, VUp, WolfStream, XVideoSharing - **XFileShare**: XFileShare based sites: Aparat, ClipWatching, GoUnlimited, GoVid, HolaVid, Streamty, TheVideoBee, Uqload, VidBom, vidlo, VidLocker, VidShare, VUp, WolfStream, XVideoSharing
- **XHamster** - **XHamster**
- **XHamsterEmbed** - **XHamsterEmbed**
- **XHamsterUser** - **XHamsterUser**
- **xiami:album**: 虾米音乐 - 专辑
- **xiami:artist**: 虾米音乐 - 歌手
- **xiami:collection**: 虾米音乐 - 精选集
- **xiami:song**: 虾米音乐
- **ximalaya**: 喜马拉雅FM - **ximalaya**: 喜马拉雅FM
- **ximalaya:album**: 喜马拉雅FM 专辑 - **ximalaya:album**: 喜马拉雅FM 专辑
- **xinpianchang**: xinpianchang.com - **xinpianchang**: xinpianchang.com
@@ -1652,6 +1709,7 @@
- **YandexVideo** - **YandexVideo**
- **YandexVideoPreview** - **YandexVideoPreview**
- **YapFiles** - **YapFiles**
- **Yappy**
- **YesJapan** - **YesJapan**
- **yinyuetai:video**: 音悦Tai - **yinyuetai:video**: 音悦Tai
- **YleAreena** - **YleAreena**
+2 -1
View File
@@ -44,5 +44,6 @@
"writesubtitles": false, "writesubtitles": false,
"allsubtitles": false, "allsubtitles": false,
"listsubtitles": false, "listsubtitles": false,
"fixup": "never" "fixup": "never",
"allow_playlist_files": false
} }
+5 -1
View File
@@ -41,7 +41,9 @@ class InfoExtractorTestRequestHandler(http.server.BaseHTTPRequestHandler):
class DummyIE(InfoExtractor): class DummyIE(InfoExtractor):
pass def _sort_formats(self, formats, field_preference=[]):
self._downloader.sort_formats(
{'formats': formats, '_format_sort_fields': field_preference})
class TestInfoExtractor(unittest.TestCase): class TestInfoExtractor(unittest.TestCase):
@@ -67,6 +69,7 @@ class TestInfoExtractor(unittest.TestCase):
<meta name="og:test1" content='foo > < bar'/> <meta name="og:test1" content='foo > < bar'/>
<meta name="og:test2" content="foo >//< bar"/> <meta name="og:test2" content="foo >//< bar"/>
<meta property=og-test3 content='Ill-formatted opengraph'/> <meta property=og-test3 content='Ill-formatted opengraph'/>
<meta property=og:test4 content=unquoted-value/>
''' '''
self.assertEqual(ie._og_search_title(html), 'Foo') self.assertEqual(ie._og_search_title(html), 'Foo')
self.assertEqual(ie._og_search_description(html), 'Some video\'s description ') self.assertEqual(ie._og_search_description(html), 'Some video\'s description ')
@@ -79,6 +82,7 @@ class TestInfoExtractor(unittest.TestCase):
self.assertEqual(ie._og_search_property(('test0', 'test1'), html), 'foo > < bar') self.assertEqual(ie._og_search_property(('test0', 'test1'), html), 'foo > < bar')
self.assertRaises(RegexNotFoundError, ie._og_search_property, 'test0', html, None, fatal=True) self.assertRaises(RegexNotFoundError, ie._og_search_property, 'test0', html, None, fatal=True)
self.assertRaises(RegexNotFoundError, ie._og_search_property, ('test0', 'test00'), html, None, fatal=True) self.assertRaises(RegexNotFoundError, ie._og_search_property, ('test0', 'test00'), html, None, fatal=True)
self.assertEqual(ie._og_search_property('test4', html), 'unquoted-value')
def test_html_search_meta(self): def test_html_search_meta(self):
ie = self.ie ie = self.ie
+17 -32
View File
@@ -68,8 +68,7 @@ class TestFormatSelection(unittest.TestCase):
{'ext': 'mp4', 'height': 460, 'url': TEST_URL}, {'ext': 'mp4', 'height': 460, 'url': TEST_URL},
] ]
info_dict = _make_result(formats) info_dict = _make_result(formats)
yie = YoutubeIE(ydl) ydl.sort_formats(info_dict)
yie._sort_formats(info_dict['formats'])
ydl.process_ie_result(info_dict) ydl.process_ie_result(info_dict)
downloaded = ydl.downloaded_info_dicts[0] downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['ext'], 'webm') self.assertEqual(downloaded['ext'], 'webm')
@@ -82,8 +81,7 @@ class TestFormatSelection(unittest.TestCase):
{'ext': 'mp4', 'height': 1080, 'url': TEST_URL}, {'ext': 'mp4', 'height': 1080, 'url': TEST_URL},
] ]
info_dict['formats'] = formats info_dict['formats'] = formats
yie = YoutubeIE(ydl) ydl.sort_formats(info_dict)
yie._sort_formats(info_dict['formats'])
ydl.process_ie_result(info_dict) ydl.process_ie_result(info_dict)
downloaded = ydl.downloaded_info_dicts[0] downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['ext'], 'mp4') self.assertEqual(downloaded['ext'], 'mp4')
@@ -97,8 +95,7 @@ class TestFormatSelection(unittest.TestCase):
{'ext': 'flv', 'height': 720, 'url': TEST_URL}, {'ext': 'flv', 'height': 720, 'url': TEST_URL},
] ]
info_dict['formats'] = formats info_dict['formats'] = formats
yie = YoutubeIE(ydl) ydl.sort_formats(info_dict)
yie._sort_formats(info_dict['formats'])
ydl.process_ie_result(info_dict) ydl.process_ie_result(info_dict)
downloaded = ydl.downloaded_info_dicts[0] downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['ext'], 'mp4') self.assertEqual(downloaded['ext'], 'mp4')
@@ -110,15 +107,14 @@ class TestFormatSelection(unittest.TestCase):
{'ext': 'webm', 'height': 720, 'url': TEST_URL}, {'ext': 'webm', 'height': 720, 'url': TEST_URL},
] ]
info_dict['formats'] = formats info_dict['formats'] = formats
yie = YoutubeIE(ydl) ydl.sort_formats(info_dict)
yie._sort_formats(info_dict['formats'])
ydl.process_ie_result(info_dict) ydl.process_ie_result(info_dict)
downloaded = ydl.downloaded_info_dicts[0] downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['ext'], 'webm') self.assertEqual(downloaded['ext'], 'webm')
def test_format_selection(self): def test_format_selection(self):
formats = [ formats = [
{'format_id': '35', 'ext': 'mp4', 'preference': 1, 'url': TEST_URL}, {'format_id': '35', 'ext': 'mp4', 'preference': 0, 'url': TEST_URL},
{'format_id': 'example-with-dashes', 'ext': 'webm', 'preference': 1, 'url': TEST_URL}, {'format_id': 'example-with-dashes', 'ext': 'webm', 'preference': 1, 'url': TEST_URL},
{'format_id': '45', 'ext': 'webm', 'preference': 2, 'url': TEST_URL}, {'format_id': '45', 'ext': 'webm', 'preference': 2, 'url': TEST_URL},
{'format_id': '47', 'ext': 'webm', 'preference': 3, 'url': TEST_URL}, {'format_id': '47', 'ext': 'webm', 'preference': 3, 'url': TEST_URL},
@@ -186,22 +182,19 @@ class TestFormatSelection(unittest.TestCase):
info_dict = _make_result(formats) info_dict = _make_result(formats)
ydl = YDL({'format': 'best'}) ydl = YDL({'format': 'best'})
ie = YoutubeIE(ydl) ydl.sort_formats(info_dict)
ie._sort_formats(info_dict['formats'])
ydl.process_ie_result(copy.deepcopy(info_dict)) ydl.process_ie_result(copy.deepcopy(info_dict))
downloaded = ydl.downloaded_info_dicts[0] downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'aac-64') self.assertEqual(downloaded['format_id'], 'aac-64')
ydl = YDL({'format': 'mp3'}) ydl = YDL({'format': 'mp3'})
ie = YoutubeIE(ydl) ydl.sort_formats(info_dict)
ie._sort_formats(info_dict['formats'])
ydl.process_ie_result(copy.deepcopy(info_dict)) ydl.process_ie_result(copy.deepcopy(info_dict))
downloaded = ydl.downloaded_info_dicts[0] downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'mp3-64') self.assertEqual(downloaded['format_id'], 'mp3-64')
ydl = YDL({'prefer_free_formats': True}) ydl = YDL({'prefer_free_formats': True})
ie = YoutubeIE(ydl) ydl.sort_formats(info_dict)
ie._sort_formats(info_dict['formats'])
ydl.process_ie_result(copy.deepcopy(info_dict)) ydl.process_ie_result(copy.deepcopy(info_dict))
downloaded = ydl.downloaded_info_dicts[0] downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], 'ogg-64') self.assertEqual(downloaded['format_id'], 'ogg-64')
@@ -346,8 +339,7 @@ class TestFormatSelection(unittest.TestCase):
info_dict = _make_result(list(formats_order), extractor='youtube') info_dict = _make_result(list(formats_order), extractor='youtube')
ydl = YDL({'format': 'bestvideo+bestaudio'}) ydl = YDL({'format': 'bestvideo+bestaudio'})
yie = YoutubeIE(ydl) ydl.sort_formats(info_dict)
yie._sort_formats(info_dict['formats'])
ydl.process_ie_result(info_dict) ydl.process_ie_result(info_dict)
downloaded = ydl.downloaded_info_dicts[0] downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], '248+172') self.assertEqual(downloaded['format_id'], '248+172')
@@ -355,40 +347,35 @@ class TestFormatSelection(unittest.TestCase):
info_dict = _make_result(list(formats_order), extractor='youtube') info_dict = _make_result(list(formats_order), extractor='youtube')
ydl = YDL({'format': 'bestvideo[height>=999999]+bestaudio/best'}) ydl = YDL({'format': 'bestvideo[height>=999999]+bestaudio/best'})
yie = YoutubeIE(ydl) ydl.sort_formats(info_dict)
yie._sort_formats(info_dict['formats'])
ydl.process_ie_result(info_dict) ydl.process_ie_result(info_dict)
downloaded = ydl.downloaded_info_dicts[0] downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], '38') self.assertEqual(downloaded['format_id'], '38')
info_dict = _make_result(list(formats_order), extractor='youtube') info_dict = _make_result(list(formats_order), extractor='youtube')
ydl = YDL({'format': 'bestvideo/best,bestaudio'}) ydl = YDL({'format': 'bestvideo/best,bestaudio'})
yie = YoutubeIE(ydl) ydl.sort_formats(info_dict)
yie._sort_formats(info_dict['formats'])
ydl.process_ie_result(info_dict) ydl.process_ie_result(info_dict)
downloaded_ids = [info['format_id'] for info in ydl.downloaded_info_dicts] downloaded_ids = [info['format_id'] for info in ydl.downloaded_info_dicts]
self.assertEqual(downloaded_ids, ['137', '141']) self.assertEqual(downloaded_ids, ['137', '141'])
info_dict = _make_result(list(formats_order), extractor='youtube') info_dict = _make_result(list(formats_order), extractor='youtube')
ydl = YDL({'format': '(bestvideo[ext=mp4],bestvideo[ext=webm])+bestaudio'}) ydl = YDL({'format': '(bestvideo[ext=mp4],bestvideo[ext=webm])+bestaudio'})
yie = YoutubeIE(ydl) ydl.sort_formats(info_dict)
yie._sort_formats(info_dict['formats'])
ydl.process_ie_result(info_dict) ydl.process_ie_result(info_dict)
downloaded_ids = [info['format_id'] for info in ydl.downloaded_info_dicts] downloaded_ids = [info['format_id'] for info in ydl.downloaded_info_dicts]
self.assertEqual(downloaded_ids, ['137+141', '248+141']) self.assertEqual(downloaded_ids, ['137+141', '248+141'])
info_dict = _make_result(list(formats_order), extractor='youtube') info_dict = _make_result(list(formats_order), extractor='youtube')
ydl = YDL({'format': '(bestvideo[ext=mp4],bestvideo[ext=webm])[height<=720]+bestaudio'}) ydl = YDL({'format': '(bestvideo[ext=mp4],bestvideo[ext=webm])[height<=720]+bestaudio'})
yie = YoutubeIE(ydl) ydl.sort_formats(info_dict)
yie._sort_formats(info_dict['formats'])
ydl.process_ie_result(info_dict) ydl.process_ie_result(info_dict)
downloaded_ids = [info['format_id'] for info in ydl.downloaded_info_dicts] downloaded_ids = [info['format_id'] for info in ydl.downloaded_info_dicts]
self.assertEqual(downloaded_ids, ['136+141', '247+141']) self.assertEqual(downloaded_ids, ['136+141', '247+141'])
info_dict = _make_result(list(formats_order), extractor='youtube') info_dict = _make_result(list(formats_order), extractor='youtube')
ydl = YDL({'format': '(bestvideo[ext=none]/bestvideo[ext=webm])+bestaudio'}) ydl = YDL({'format': '(bestvideo[ext=none]/bestvideo[ext=webm])+bestaudio'})
yie = YoutubeIE(ydl) ydl.sort_formats(info_dict)
yie._sort_formats(info_dict['formats'])
ydl.process_ie_result(info_dict) ydl.process_ie_result(info_dict)
downloaded_ids = [info['format_id'] for info in ydl.downloaded_info_dicts] downloaded_ids = [info['format_id'] for info in ydl.downloaded_info_dicts]
self.assertEqual(downloaded_ids, ['248+141']) self.assertEqual(downloaded_ids, ['248+141'])
@@ -396,16 +383,14 @@ class TestFormatSelection(unittest.TestCase):
for f1, f2 in zip(formats_order, formats_order[1:]): for f1, f2 in zip(formats_order, formats_order[1:]):
info_dict = _make_result([f1, f2], extractor='youtube') info_dict = _make_result([f1, f2], extractor='youtube')
ydl = YDL({'format': 'best/bestvideo'}) ydl = YDL({'format': 'best/bestvideo'})
yie = YoutubeIE(ydl) ydl.sort_formats(info_dict)
yie._sort_formats(info_dict['formats'])
ydl.process_ie_result(info_dict) ydl.process_ie_result(info_dict)
downloaded = ydl.downloaded_info_dicts[0] downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], f1['format_id']) self.assertEqual(downloaded['format_id'], f1['format_id'])
info_dict = _make_result([f2, f1], extractor='youtube') info_dict = _make_result([f2, f1], extractor='youtube')
ydl = YDL({'format': 'best/bestvideo'}) ydl = YDL({'format': 'best/bestvideo'})
yie = YoutubeIE(ydl) ydl.sort_formats(info_dict)
yie._sort_formats(info_dict['formats'])
ydl.process_ie_result(info_dict) ydl.process_ie_result(info_dict)
downloaded = ydl.downloaded_info_dicts[0] downloaded = ydl.downloaded_info_dicts[0]
self.assertEqual(downloaded['format_id'], f1['format_id']) self.assertEqual(downloaded['format_id'], f1['format_id'])
@@ -480,7 +465,7 @@ class TestFormatSelection(unittest.TestCase):
for f in formats: for f in formats:
f['url'] = 'http://_/' f['url'] = 'http://_/'
f['ext'] = 'unknown' f['ext'] = 'unknown'
info_dict = _make_result(formats) info_dict = _make_result(formats, _format_sort_fields=('id', ))
ydl = YDL({'format': 'best[filesize<3000]'}) ydl = YDL({'format': 'best[filesize<3000]'})
ydl.process_ie_result(info_dict) ydl.process_ie_result(info_dict)
+3 -3
View File
@@ -26,7 +26,7 @@ from yt_dlp.aes import (
key_expansion, key_expansion,
pad_block, pad_block,
) )
from yt_dlp.dependencies import Cryptodome_AES from yt_dlp.dependencies import Cryptodome
from yt_dlp.utils import bytes_to_intlist, intlist_to_bytes from yt_dlp.utils import bytes_to_intlist, intlist_to_bytes
# the encrypted data can be generate with 'devscripts/generate_aes_testdata.py' # the encrypted data can be generate with 'devscripts/generate_aes_testdata.py'
@@ -48,7 +48,7 @@ class TestAES(unittest.TestCase):
data = b'\x97\x92+\xe5\x0b\xc3\x18\x91ky9m&\xb3\xb5@\xe6\x27\xc2\x96.\xc8u\x88\xab9-[\x9e|\xf1\xcd' data = b'\x97\x92+\xe5\x0b\xc3\x18\x91ky9m&\xb3\xb5@\xe6\x27\xc2\x96.\xc8u\x88\xab9-[\x9e|\xf1\xcd'
decrypted = intlist_to_bytes(aes_cbc_decrypt(bytes_to_intlist(data), self.key, self.iv)) decrypted = intlist_to_bytes(aes_cbc_decrypt(bytes_to_intlist(data), self.key, self.iv))
self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg) self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg)
if Cryptodome_AES: if Cryptodome:
decrypted = aes_cbc_decrypt_bytes(data, intlist_to_bytes(self.key), intlist_to_bytes(self.iv)) decrypted = aes_cbc_decrypt_bytes(data, intlist_to_bytes(self.key), intlist_to_bytes(self.iv))
self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg) self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg)
@@ -78,7 +78,7 @@ class TestAES(unittest.TestCase):
decrypted = intlist_to_bytes(aes_gcm_decrypt_and_verify( decrypted = intlist_to_bytes(aes_gcm_decrypt_and_verify(
bytes_to_intlist(data), self.key, bytes_to_intlist(authentication_tag), self.iv[:12])) bytes_to_intlist(data), self.key, bytes_to_intlist(authentication_tag), self.iv[:12]))
self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg) self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg)
if Cryptodome_AES: if Cryptodome:
decrypted = aes_gcm_decrypt_and_verify_bytes( decrypted = aes_gcm_decrypt_and_verify_bytes(
data, intlist_to_bytes(self.key), authentication_tag, intlist_to_bytes(self.iv[:12])) data, intlist_to_bytes(self.key), authentication_tag, intlist_to_bytes(self.iv[:12]))
self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg) self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg)
+10 -5
View File
@@ -10,6 +10,7 @@ sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from test.helper import is_download_test, try_rm from test.helper import is_download_test, try_rm
from yt_dlp import YoutubeDL from yt_dlp import YoutubeDL
from yt_dlp.utils import DownloadError
def _download_restricted(url, filename, age): def _download_restricted(url, filename, age):
@@ -25,10 +26,14 @@ def _download_restricted(url, filename, age):
ydl.add_default_info_extractors() ydl.add_default_info_extractors()
json_filename = os.path.splitext(filename)[0] + '.info.json' json_filename = os.path.splitext(filename)[0] + '.info.json'
try_rm(json_filename) try_rm(json_filename)
try:
ydl.download([url]) ydl.download([url])
res = os.path.exists(json_filename) except DownloadError:
pass
else:
return os.path.exists(json_filename)
finally:
try_rm(json_filename) try_rm(json_filename)
return res
@is_download_test @is_download_test
@@ -38,12 +43,12 @@ class TestAgeRestriction(unittest.TestCase):
self.assertFalse(_download_restricted(url, filename, age)) self.assertFalse(_download_restricted(url, filename, age))
def test_youtube(self): def test_youtube(self):
self._assert_restricted('07FYdnEawAQ', '07FYdnEawAQ.mp4', 10) self._assert_restricted('HtVdAasjOgU', 'HtVdAasjOgU.mp4', 10)
def test_youporn(self): def test_youporn(self):
self._assert_restricted( self._assert_restricted(
'http://www.youporn.com/watch/505835/sex-ed-is-it-safe-to-masturbate-daily/', 'https://www.youporn.com/watch/16715086/sex-ed-in-detention-18-asmr/',
'505835.mp4', 2, old_age=25) '16715086.mp4', 2, old_age=25)
if __name__ == '__main__': if __name__ == '__main__':
+3
View File
@@ -31,6 +31,9 @@ class TestCompat(unittest.TestCase):
# TODO: Test submodule # TODO: Test submodule
# compat.asyncio.events # Must not raise error # compat.asyncio.events # Must not raise error
with self.assertWarns(DeprecationWarning):
compat.compat_pycrypto_AES # Must not raise error
def test_compat_expanduser(self): def test_compat_expanduser(self):
old_home = os.environ.get('HOME') old_home = os.environ.get('HOME')
test_str = R'C:\Documents and Settings\тест\Application Data' test_str = R'C:\Documents and Settings\тест\Application Data'
+227
View File
@@ -0,0 +1,227 @@
#!/usr/bin/env python3
# Allow direct execution
import os
import sys
import unittest
import unittest.mock
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import contextlib
import itertools
from pathlib import Path
from yt_dlp.compat import compat_expanduser
from yt_dlp.options import create_parser, parseOpts
from yt_dlp.utils import Config, get_executable_path
ENVIRON_DEFAULTS = {
'HOME': None,
'XDG_CONFIG_HOME': '/_xdg_config_home/',
'USERPROFILE': 'C:/Users/testing/',
'APPDATA': 'C:/Users/testing/AppData/Roaming/',
'HOMEDRIVE': 'C:/',
'HOMEPATH': 'Users/testing/',
}
@contextlib.contextmanager
def set_environ(**kwargs):
saved_environ = os.environ.copy()
for name, value in {**ENVIRON_DEFAULTS, **kwargs}.items():
if value is None:
os.environ.pop(name, None)
else:
os.environ[name] = value
yield
os.environ.clear()
os.environ.update(saved_environ)
def _generate_expected_groups():
xdg_config_home = os.getenv('XDG_CONFIG_HOME') or compat_expanduser('~/.config')
appdata_dir = os.getenv('appdata')
home_dir = compat_expanduser('~')
return {
'Portable': [
Path(get_executable_path(), 'yt-dlp.conf'),
],
'Home': [
Path('yt-dlp.conf'),
],
'User': [
Path(xdg_config_home, 'yt-dlp.conf'),
Path(xdg_config_home, 'yt-dlp', 'config'),
Path(xdg_config_home, 'yt-dlp', 'config.txt'),
*((
Path(appdata_dir, 'yt-dlp.conf'),
Path(appdata_dir, 'yt-dlp', 'config'),
Path(appdata_dir, 'yt-dlp', 'config.txt'),
) if appdata_dir else ()),
Path(home_dir, 'yt-dlp.conf'),
Path(home_dir, 'yt-dlp.conf.txt'),
Path(home_dir, '.yt-dlp', 'config'),
Path(home_dir, '.yt-dlp', 'config.txt'),
],
'System': [
Path('/etc/yt-dlp.conf'),
Path('/etc/yt-dlp/config'),
Path('/etc/yt-dlp/config.txt'),
]
}
class TestConfig(unittest.TestCase):
maxDiff = None
@set_environ()
def test_config__ENVIRON_DEFAULTS_sanity(self):
expected = make_expected()
self.assertCountEqual(
set(expected), expected,
'ENVIRON_DEFAULTS produces non unique names')
def test_config_all_environ_values(self):
for name, value in ENVIRON_DEFAULTS.items():
for new_value in (None, '', '.', value or '/some/dir'):
with set_environ(**{name: new_value}):
self._simple_grouping_test()
def test_config_default_expected_locations(self):
files, _ = self._simple_config_test()
self.assertEqual(
files, make_expected(),
'Not all expected locations have been checked')
def test_config_default_grouping(self):
self._simple_grouping_test()
def _simple_grouping_test(self):
expected_groups = make_expected_groups()
for name, group in expected_groups.items():
for index, existing_path in enumerate(group):
result, opts = self._simple_config_test(existing_path)
expected = expected_from_expected_groups(expected_groups, existing_path)
self.assertEqual(
result, expected,
f'The checked locations do not match the expected ({name}, {index})')
self.assertEqual(
opts.outtmpl['default'], '1',
f'The used result value was incorrect ({name}, {index})')
def _simple_config_test(self, *stop_paths):
encountered = 0
paths = []
def read_file(filename, default=[]):
nonlocal encountered
path = Path(filename)
paths.append(path)
if path in stop_paths:
encountered += 1
return ['-o', f'{encountered}']
with ConfigMock(read_file):
_, opts, _ = parseOpts([], False)
return paths, opts
@set_environ()
def test_config_early_exit_commandline(self):
self._early_exit_test(0, '--ignore-config')
@set_environ()
def test_config_early_exit_files(self):
for index, _ in enumerate(make_expected(), 1):
self._early_exit_test(index)
def _early_exit_test(self, allowed_reads, *args):
reads = 0
def read_file(filename, default=[]):
nonlocal reads
reads += 1
if reads > allowed_reads:
self.fail('The remaining config was not ignored')
elif reads == allowed_reads:
return ['--ignore-config']
with ConfigMock(read_file):
parseOpts(args, False)
@set_environ()
def test_config_override_commandline(self):
self._override_test(0, '-o', 'pass')
@set_environ()
def test_config_override_files(self):
for index, _ in enumerate(make_expected(), 1):
self._override_test(index)
def _override_test(self, start_index, *args):
index = 0
def read_file(filename, default=[]):
nonlocal index
index += 1
if index > start_index:
return ['-o', 'fail']
elif index == start_index:
return ['-o', 'pass']
with ConfigMock(read_file):
_, opts, _ = parseOpts(args, False)
self.assertEqual(
opts.outtmpl['default'], 'pass',
'The earlier group did not override the later ones')
@contextlib.contextmanager
def ConfigMock(read_file=None):
with unittest.mock.patch('yt_dlp.options.Config') as mock:
mock.return_value = Config(create_parser())
if read_file is not None:
mock.read_file = read_file
yield mock
def make_expected(*filepaths):
return expected_from_expected_groups(_generate_expected_groups(), *filepaths)
def make_expected_groups(*filepaths):
return _filter_expected_groups(_generate_expected_groups(), filepaths)
def expected_from_expected_groups(expected_groups, *filepaths):
return list(itertools.chain.from_iterable(
_filter_expected_groups(expected_groups, filepaths).values()))
def _filter_expected_groups(expected, filepaths):
if not filepaths:
return expected
result = {}
for group, paths in expected.items():
new_paths = []
for path in paths:
new_paths.append(path)
if path in filepaths:
break
result[group] = new_paths
return result
if __name__ == '__main__':
unittest.main()
+32
View File
@@ -155,6 +155,38 @@ class TestJSInterpreter(unittest.TestCase):
self.assertEqual(jsi.call_function('z'), 5) self.assertEqual(jsi.call_function('z'), 5)
self.assertEqual(jsi.call_function('y'), 2) self.assertEqual(jsi.call_function('y'), 2)
def test_if(self):
jsi = JSInterpreter('''
function x() {
let a = 9;
if (0==0) {a++}
return a
}''')
self.assertEqual(jsi.call_function('x'), 10)
jsi = JSInterpreter('''
function x() {
if (0==0) {return 10}
}''')
self.assertEqual(jsi.call_function('x'), 10)
jsi = JSInterpreter('''
function x() {
if (0!=0) {return 1}
else {return 10}
}''')
self.assertEqual(jsi.call_function('x'), 10)
""" # Unsupported
jsi = JSInterpreter('''
function x() {
if (0!=0) {return 1}
else if (1==0) {return 2}
else {return 10}
}''')
self.assertEqual(jsi.call_function('x'), 10)
"""
def test_for_loop(self): def test_for_loop(self):
jsi = JSInterpreter(''' jsi = JSInterpreter('''
function x() { a=0; for (i=0; i-10; i++) {a++} return a } function x() { a=0; for (i=0; i-10; i++) {a++} return a }
+73
View File
@@ -0,0 +1,73 @@
import importlib
import os
import shutil
import sys
import unittest
from pathlib import Path
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
TEST_DATA_DIR = Path(os.path.dirname(os.path.abspath(__file__)), 'testdata')
sys.path.append(str(TEST_DATA_DIR))
importlib.invalidate_caches()
from yt_dlp.plugins import PACKAGE_NAME, directories, load_plugins
class TestPlugins(unittest.TestCase):
TEST_PLUGIN_DIR = TEST_DATA_DIR / PACKAGE_NAME
def test_directories_containing_plugins(self):
self.assertIn(self.TEST_PLUGIN_DIR, map(Path, directories()))
def test_extractor_classes(self):
for module_name in tuple(sys.modules):
if module_name.startswith(f'{PACKAGE_NAME}.extractor'):
del sys.modules[module_name]
plugins_ie = load_plugins('extractor', 'IE')
self.assertIn(f'{PACKAGE_NAME}.extractor.normal', sys.modules.keys())
self.assertIn('NormalPluginIE', plugins_ie.keys())
# don't load modules with underscore prefix
self.assertFalse(
f'{PACKAGE_NAME}.extractor._ignore' in sys.modules.keys(),
'loaded module beginning with underscore')
self.assertNotIn('IgnorePluginIE', plugins_ie.keys())
# Don't load extractors with underscore prefix
self.assertNotIn('_IgnoreUnderscorePluginIE', plugins_ie.keys())
# Don't load extractors not specified in __all__ (if supplied)
self.assertNotIn('IgnoreNotInAllPluginIE', plugins_ie.keys())
self.assertIn('InAllPluginIE', plugins_ie.keys())
def test_postprocessor_classes(self):
plugins_pp = load_plugins('postprocessor', 'PP')
self.assertIn('NormalPluginPP', plugins_pp.keys())
def test_importing_zipped_module(self):
zip_path = TEST_DATA_DIR / 'zipped_plugins.zip'
shutil.make_archive(str(zip_path)[:-4], 'zip', str(zip_path)[:-4])
sys.path.append(str(zip_path)) # add zip to search paths
importlib.invalidate_caches() # reset the import caches
try:
for plugin_type in ('extractor', 'postprocessor'):
package = importlib.import_module(f'{PACKAGE_NAME}.{plugin_type}')
self.assertIn(zip_path / PACKAGE_NAME / plugin_type, map(Path, package.__path__))
plugins_ie = load_plugins('extractor', 'IE')
self.assertIn('ZippedPluginIE', plugins_ie.keys())
plugins_pp = load_plugins('postprocessor', 'PP')
self.assertIn('ZippedPluginPP', plugins_pp.keys())
finally:
sys.path.remove(str(zip_path))
os.remove(zip_path)
importlib.invalidate_caches() # reset the import caches
if __name__ == '__main__':
unittest.main()
+180 -27
View File
@@ -105,6 +105,7 @@ from yt_dlp.utils import (
sanitized_Request, sanitized_Request,
shell_quote, shell_quote,
smuggle_url, smuggle_url,
str_or_none,
str_to_int, str_to_int,
strip_jsonp, strip_jsonp,
strip_or_none, strip_or_none,
@@ -954,6 +955,85 @@ class TestUtil(unittest.TestCase):
) )
self.assertEqual(escape_url('http://vimeo.com/56015672#at=0'), 'http://vimeo.com/56015672#at=0') self.assertEqual(escape_url('http://vimeo.com/56015672#at=0'), 'http://vimeo.com/56015672#at=0')
def test_js_to_json_vars_strings(self):
self.assertDictEqual(
json.loads(js_to_json(
'''{
'null': a,
'nullStr': b,
'true': c,
'trueStr': d,
'false': e,
'falseStr': f,
'unresolvedVar': g,
}''',
{
'a': 'null',
'b': '"null"',
'c': 'true',
'd': '"true"',
'e': 'false',
'f': '"false"',
'g': 'var',
}
)),
{
'null': None,
'nullStr': 'null',
'true': True,
'trueStr': 'true',
'false': False,
'falseStr': 'false',
'unresolvedVar': 'var'
}
)
self.assertDictEqual(
json.loads(js_to_json(
'''{
'int': a,
'intStr': b,
'float': c,
'floatStr': d,
}''',
{
'a': '123',
'b': '"123"',
'c': '1.23',
'd': '"1.23"',
}
)),
{
'int': 123,
'intStr': '123',
'float': 1.23,
'floatStr': '1.23',
}
)
self.assertDictEqual(
json.loads(js_to_json(
'''{
'object': a,
'objectStr': b,
'array': c,
'arrayStr': d,
}''',
{
'a': '{}',
'b': '"{}"',
'c': '[]',
'd': '"[]"',
}
)),
{
'object': {},
'objectStr': '{}',
'array': [],
'arrayStr': '[]',
}
)
def test_js_to_json_realworld(self): def test_js_to_json_realworld(self):
inp = '''{ inp = '''{
'clip':{'provider':'pseudo'} 'clip':{'provider':'pseudo'}
@@ -1874,6 +1954,8 @@ Line 1
vcodecs=[None], acodecs=[None], vexts=['webm'], aexts=['m4a']), 'mkv') vcodecs=[None], acodecs=[None], vexts=['webm'], aexts=['m4a']), 'mkv')
self.assertEqual(get_compatible_ext( self.assertEqual(get_compatible_ext(
vcodecs=[None], acodecs=[None], vexts=['webm'], aexts=['webm']), 'webm') vcodecs=[None], acodecs=[None], vexts=['webm'], aexts=['webm']), 'webm')
self.assertEqual(get_compatible_ext(
vcodecs=[None], acodecs=[None], vexts=['webm'], aexts=['weba']), 'webm')
self.assertEqual(get_compatible_ext( self.assertEqual(get_compatible_ext(
vcodecs=['h264'], acodecs=['mp4a'], vexts=['mov'], aexts=['m4a']), 'mp4') vcodecs=['h264'], acodecs=['mp4a'], vexts=['mov'], aexts=['m4a']), 'mp4')
@@ -1918,8 +2000,8 @@ Line 1
# Test Ellipsis behavior # Test Ellipsis behavior
self.assertCountEqual(traverse_obj(_TEST_DATA, ...), self.assertCountEqual(traverse_obj(_TEST_DATA, ...),
(item for item in _TEST_DATA.values() if item is not None), (item for item in _TEST_DATA.values() if item not in (None, {})),
msg='`...` should give all values except `None`') msg='`...` should give all non discarded values')
self.assertCountEqual(traverse_obj(_TEST_DATA, ('urls', 0, ...)), _TEST_DATA['urls'][0].values(), self.assertCountEqual(traverse_obj(_TEST_DATA, ('urls', 0, ...)), _TEST_DATA['urls'][0].values(),
msg='`...` selection for dicts should select all values') msg='`...` selection for dicts should select all values')
self.assertEqual(traverse_obj(_TEST_DATA, (..., ..., 'url')), self.assertEqual(traverse_obj(_TEST_DATA, (..., ..., 'url')),
@@ -1934,6 +2016,29 @@ Line 1
msg='function as query key should perform a filter based on (key, value)') msg='function as query key should perform a filter based on (key, value)')
self.assertCountEqual(traverse_obj(_TEST_DATA, lambda _, x: isinstance(x[0], str)), {'str'}, self.assertCountEqual(traverse_obj(_TEST_DATA, lambda _, x: isinstance(x[0], str)), {'str'},
msg='exceptions in the query function should be catched') msg='exceptions in the query function should be catched')
if __debug__:
with self.assertRaises(Exception, msg='Wrong function signature should raise in debug'):
traverse_obj(_TEST_DATA, lambda a: ...)
with self.assertRaises(Exception, msg='Wrong function signature should raise in debug'):
traverse_obj(_TEST_DATA, lambda a, b, c: ...)
# Test set as key (transformation/type, like `expected_type`)
self.assertEqual(traverse_obj(_TEST_DATA, (..., {str.upper}, )), ['STR'],
msg='Function in set should be a transformation')
self.assertEqual(traverse_obj(_TEST_DATA, (..., {str})), ['str'],
msg='Type in set should be a type filter')
self.assertEqual(traverse_obj(_TEST_DATA, {dict}), _TEST_DATA,
msg='A single set should be wrapped into a path')
self.assertEqual(traverse_obj(_TEST_DATA, (..., {str.upper})), ['STR'],
msg='Transformation function should not raise')
self.assertEqual(traverse_obj(_TEST_DATA, (..., {str_or_none})),
[item for item in map(str_or_none, _TEST_DATA.values()) if item is not None],
msg='Function in set should be a transformation')
if __debug__:
with self.assertRaises(Exception, msg='Sets with length != 1 should raise in debug'):
traverse_obj(_TEST_DATA, set())
with self.assertRaises(Exception, msg='Sets with length != 1 should raise in debug'):
traverse_obj(_TEST_DATA, {str.upper, str})
# Test alternative paths # Test alternative paths
self.assertEqual(traverse_obj(_TEST_DATA, 'fail', 'str'), 'str', self.assertEqual(traverse_obj(_TEST_DATA, 'fail', 'str'), 'str',
@@ -1979,15 +2084,23 @@ Line 1
{0: ['https://www.example.com/1', 'https://www.example.com/0']}, {0: ['https://www.example.com/1', 'https://www.example.com/0']},
msg='tripple nesting in dict path should be treated as branches') msg='tripple nesting in dict path should be treated as branches')
self.assertEqual(traverse_obj(_TEST_DATA, {0: 'fail'}), {}, self.assertEqual(traverse_obj(_TEST_DATA, {0: 'fail'}), {},
msg='remove `None` values when dict key') msg='remove `None` values when top level dict key fails')
self.assertEqual(traverse_obj(_TEST_DATA, {0: 'fail'}, default=...), {0: ...}, self.assertEqual(traverse_obj(_TEST_DATA, {0: 'fail'}, default=...), {0: ...},
msg='do not remove `None` values if `default`') msg='use `default` if key fails and `default`')
self.assertEqual(traverse_obj(_TEST_DATA, {0: 'dict'}), {0: {}}, self.assertEqual(traverse_obj(_TEST_DATA, {0: 'dict'}), {},
msg='do not remove empty values when dict key') msg='remove empty values when dict key')
self.assertEqual(traverse_obj(_TEST_DATA, {0: 'dict'}, default=...), {0: {}}, self.assertEqual(traverse_obj(_TEST_DATA, {0: 'dict'}, default=...), {0: ...},
msg='do not remove empty values when dict key and a default') msg='use `default` when dict key and `default`')
self.assertEqual(traverse_obj(_TEST_DATA, {0: ('dict', ...)}), {0: []}, self.assertEqual(traverse_obj(_TEST_DATA, {0: {0: 'fail'}}), {},
msg='if branch in dict key not successful, return `[]`') msg='remove empty values when nested dict key fails')
self.assertEqual(traverse_obj(None, {0: 'fail'}), {},
msg='default to dict if pruned')
self.assertEqual(traverse_obj(None, {0: 'fail'}, default=...), {0: ...},
msg='default to dict if pruned and default is given')
self.assertEqual(traverse_obj(_TEST_DATA, {0: {0: 'fail'}}, default=...), {0: {0: ...}},
msg='use nested `default` when nested dict key fails and `default`')
self.assertEqual(traverse_obj(_TEST_DATA, {0: ('dict', ...)}), {},
msg='remove key if branch in dict key not successful')
# Testing default parameter behavior # Testing default parameter behavior
_DEFAULT_DATA = {'None': None, 'int': 0, 'list': []} _DEFAULT_DATA = {'None': None, 'int': 0, 'list': []}
@@ -2011,20 +2124,55 @@ Line 1
msg='if branched but not successful return `[]`, not `default`') msg='if branched but not successful return `[]`, not `default`')
self.assertEqual(traverse_obj(_DEFAULT_DATA, ('list', ...)), [], self.assertEqual(traverse_obj(_DEFAULT_DATA, ('list', ...)), [],
msg='if branched but object is empty return `[]`, not `default`') msg='if branched but object is empty return `[]`, not `default`')
self.assertEqual(traverse_obj(None, ...), [],
msg='if branched but object is `None` return `[]`, not `default`')
self.assertEqual(traverse_obj({0: None}, (0, ...)), [],
msg='if branched but state is `None` return `[]`, not `default`')
branching_paths = [
('fail', ...),
(..., 'fail'),
100 * ('fail',) + (...,),
(...,) + 100 * ('fail',),
]
for branching_path in branching_paths:
self.assertEqual(traverse_obj({}, branching_path), [],
msg='if branched but state is `None`, return `[]` (not `default`)')
self.assertEqual(traverse_obj({}, 'fail', branching_path), [],
msg='if branching in last alternative and previous did not match, return `[]` (not `default`)')
self.assertEqual(traverse_obj({0: 'x'}, 0, branching_path), 'x',
msg='if branching in last alternative and previous did match, return single value')
self.assertEqual(traverse_obj({0: 'x'}, branching_path, 0), 'x',
msg='if branching in first alternative and non-branching path does match, return single value')
self.assertEqual(traverse_obj({}, branching_path, 'fail'), None,
msg='if branching in first alternative and non-branching path does not match, return `default`')
# Testing expected_type behavior # Testing expected_type behavior
_EXPECTED_TYPE_DATA = {'str': 'str', 'int': 0} _EXPECTED_TYPE_DATA = {'str': 'str', 'int': 0}
self.assertEqual(traverse_obj(_EXPECTED_TYPE_DATA, 'str', expected_type=str), 'str', self.assertEqual(traverse_obj(_EXPECTED_TYPE_DATA, 'str', expected_type=str),
msg='accept matching `expected_type` type') 'str', msg='accept matching `expected_type` type')
self.assertEqual(traverse_obj(_EXPECTED_TYPE_DATA, 'str', expected_type=int), None, self.assertEqual(traverse_obj(_EXPECTED_TYPE_DATA, 'str', expected_type=int),
msg='reject non matching `expected_type` type') None, msg='reject non matching `expected_type` type')
self.assertEqual(traverse_obj(_EXPECTED_TYPE_DATA, 'int', expected_type=lambda x: str(x)), '0', self.assertEqual(traverse_obj(_EXPECTED_TYPE_DATA, 'int', expected_type=lambda x: str(x)),
msg='transform type using type function') '0', msg='transform type using type function')
self.assertEqual(traverse_obj(_EXPECTED_TYPE_DATA, 'str', self.assertEqual(traverse_obj(_EXPECTED_TYPE_DATA, 'str', expected_type=lambda _: 1 / 0),
expected_type=lambda _: 1 / 0), None, None, msg='wrap expected_type fuction in try_call')
msg='wrap expected_type fuction in try_call') self.assertEqual(traverse_obj(_EXPECTED_TYPE_DATA, ..., expected_type=str),
self.assertEqual(traverse_obj(_EXPECTED_TYPE_DATA, ..., expected_type=str), ['str'], ['str'], msg='eliminate items that expected_type fails on')
msg='eliminate items that expected_type fails on') self.assertEqual(traverse_obj(_TEST_DATA, {0: 100, 1: 1.2}, expected_type=int),
{0: 100}, msg='type as expected_type should filter dict values')
self.assertEqual(traverse_obj(_TEST_DATA, {0: 100, 1: 1.2, 2: 'None'}, expected_type=str_or_none),
{0: '100', 1: '1.2'}, msg='function as expected_type should transform dict values')
self.assertEqual(traverse_obj(_TEST_DATA, ({0: 1.2}, 0, {int_or_none}), expected_type=int),
1, msg='expected_type should not filter non final dict values')
self.assertEqual(traverse_obj(_TEST_DATA, {0: {0: 100, 1: 'str'}}, expected_type=int),
{0: {0: 100}}, msg='expected_type should transform deep dict values')
self.assertEqual(traverse_obj(_TEST_DATA, [({0: '...'}, {0: '...'})], expected_type=type(...)),
[{0: ...}, {0: ...}], msg='expected_type should transform branched dict values')
self.assertEqual(traverse_obj({1: {3: 4}}, [(1, 2), 3], expected_type=int),
[4], msg='expected_type regression for type matching in tuple branching')
self.assertEqual(traverse_obj(_TEST_DATA, ['data', ...], expected_type=int),
[], msg='expected_type regression for type matching in dict result')
# Test get_all behavior # Test get_all behavior
_GET_ALL_DATA = {'key': [0, 1, 2]} _GET_ALL_DATA = {'key': [0, 1, 2]}
@@ -2064,14 +2212,17 @@ Line 1
traverse_string=True), '.', traverse_string=True), '.',
msg='traverse into converted data if `traverse_string`') msg='traverse into converted data if `traverse_string`')
self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', ...), self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', ...),
traverse_string=True), list('str'), traverse_string=True), 'str',
msg='`...` branching into string should result in list') msg='`...` should result in string (same value) if `traverse_string`')
self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', slice(0, None, 2)),
traverse_string=True), 'sr',
msg='`slice` should result in string if `traverse_string`')
self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', lambda i, v: i or v == "s"),
traverse_string=True), 'str',
msg='function should result in string if `traverse_string`')
self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', (0, 2)), self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', (0, 2)),
traverse_string=True), ['s', 'r'], traverse_string=True), ['s', 'r'],
msg='branching into string should result in list') msg='branching should result in list if `traverse_string`')
self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', lambda _, x: x),
traverse_string=True), list('str'),
msg='function branching into string should result in list')
# Test is_user_input behavior # Test is_user_input behavior
_IS_USER_INPUT_DATA = {'range8': list(range(8))} _IS_USER_INPUT_DATA = {'range8': list(range(8))}
@@ -2108,6 +2259,8 @@ Line 1
msg='failing str key on a `re.Match` should return `default`') msg='failing str key on a `re.Match` should return `default`')
self.assertEqual(traverse_obj(mobj, 8), None, self.assertEqual(traverse_obj(mobj, 8), None,
msg='failing int key on a `re.Match` should return `default`') msg='failing int key on a `re.Match` should return `default`')
self.assertEqual(traverse_obj(mobj, lambda k, _: k in (0, 'group')), ['0123', '3'],
msg='function on a `re.Match` should give group name as well')
if __name__ == '__main__': if __name__ == '__main__':
+4
View File
@@ -134,6 +134,10 @@ _NSIG_TESTS = [
'https://www.youtube.com/s/player/7a062b77/player_ias.vflset/en_US/base.js', 'https://www.youtube.com/s/player/7a062b77/player_ias.vflset/en_US/base.js',
'NRcE3y3mVtm_cV-W', 'VbsCYUATvqlt5w', 'NRcE3y3mVtm_cV-W', 'VbsCYUATvqlt5w',
), ),
(
'https://www.youtube.com/s/player/dac945fd/player_ias.vflset/en_US/base.js',
'o8BkRxXhuYsBCWi6RplPdP', '3Lx32v_hmzTm6A',
),
] ]
+5
View File
@@ -0,0 +1,5 @@
from yt_dlp.extractor.common import InfoExtractor
class IgnorePluginIE(InfoExtractor):
pass
+12
View File
@@ -0,0 +1,12 @@
from yt_dlp.extractor.common import InfoExtractor
class IgnoreNotInAllPluginIE(InfoExtractor):
pass
class InAllPluginIE(InfoExtractor):
pass
__all__ = ['InAllPluginIE']
+9
View File
@@ -0,0 +1,9 @@
from yt_dlp.extractor.common import InfoExtractor
class NormalPluginIE(InfoExtractor):
pass
class _IgnoreUnderscorePluginIE(InfoExtractor):
pass
+5
View File
@@ -0,0 +1,5 @@
from yt_dlp.postprocessor.common import PostProcessor
class NormalPluginPP(PostProcessor):
pass
@@ -0,0 +1,5 @@
from yt_dlp.extractor.common import InfoExtractor
class ZippedPluginIE(InfoExtractor):
pass
@@ -0,0 +1,5 @@
from yt_dlp.postprocessor.common import PostProcessor
class ZippedPluginPP(PostProcessor):
pass
+129 -71
View File
@@ -32,7 +32,8 @@ from .extractor import gen_extractor_classes, get_info_extractor
from .extractor.common import UnsupportedURLIE from .extractor.common import UnsupportedURLIE
from .extractor.openload import PhantomJSwrapper from .extractor.openload import PhantomJSwrapper
from .minicurses import format_text from .minicurses import format_text
from .postprocessor import _PLUGIN_CLASSES as plugin_postprocessors from .plugins import directories as plugin_directories
from .postprocessor import _PLUGIN_CLASSES as plugin_pps
from .postprocessor import ( from .postprocessor import (
EmbedThumbnailPP, EmbedThumbnailPP,
FFmpegFixupDuplicateMoovPP, FFmpegFixupDuplicateMoovPP,
@@ -67,6 +68,7 @@ from .utils import (
EntryNotInPlaylist, EntryNotInPlaylist,
ExistingVideoReached, ExistingVideoReached,
ExtractorError, ExtractorError,
FormatSorter,
GeoRestrictedError, GeoRestrictedError,
HEADRequest, HEADRequest,
ISO3166Utils, ISO3166Utils,
@@ -316,6 +318,7 @@ class YoutubeDL:
If not provided and the key is encrypted, yt-dlp will ask interactively If not provided and the key is encrypted, yt-dlp will ask interactively
prefer_insecure: Use HTTP instead of HTTPS to retrieve information. prefer_insecure: Use HTTP instead of HTTPS to retrieve information.
(Only supported by some extractors) (Only supported by some extractors)
enable_file_urls: Enable file:// URLs. This is disabled by default for security reasons.
http_headers: A dictionary of custom headers to be used for all requests http_headers: A dictionary of custom headers to be used for all requests
proxy: URL of the proxy server to use proxy: URL of the proxy server to use
geo_verification_proxy: URL of the proxy to use for IP address verification geo_verification_proxy: URL of the proxy to use for IP address verification
@@ -547,11 +550,11 @@ class YoutubeDL:
_format_fields = { _format_fields = {
# NB: Keep in sync with the docstring of extractor/common.py # NB: Keep in sync with the docstring of extractor/common.py
'url', 'manifest_url', 'manifest_stream_number', 'ext', 'format', 'format_id', 'format_note', 'url', 'manifest_url', 'manifest_stream_number', 'ext', 'format', 'format_id', 'format_note',
'width', 'height', 'resolution', 'dynamic_range', 'tbr', 'abr', 'acodec', 'asr', 'audio_channels', 'width', 'height', 'aspect_ratio', 'resolution', 'dynamic_range', 'tbr', 'abr', 'acodec', 'asr', 'audio_channels',
'vbr', 'fps', 'vcodec', 'container', 'filesize', 'filesize_approx', 'rows', 'columns', 'vbr', 'fps', 'vcodec', 'container', 'filesize', 'filesize_approx', 'rows', 'columns',
'player_url', 'protocol', 'fragment_base_url', 'fragments', 'is_from_start', 'player_url', 'protocol', 'fragment_base_url', 'fragments', 'is_from_start',
'preference', 'language', 'language_preference', 'quality', 'source_preference', 'preference', 'language', 'language_preference', 'quality', 'source_preference',
'http_headers', 'stretched_ratio', 'no_resume', 'has_drm', 'downloader_options', 'http_headers', 'stretched_ratio', 'no_resume', 'has_drm', 'extra_param_to_segment_url', 'hls_aes', 'downloader_options',
'page_url', 'app', 'play_path', 'tc_url', 'flash_version', 'rtmp_live', 'rtmp_conn', 'rtmp_protocol', 'rtmp_real_time' 'page_url', 'app', 'play_path', 'tc_url', 'flash_version', 'rtmp_live', 'rtmp_conn', 'rtmp_protocol', 'rtmp_real_time'
} }
_format_selection_exts = { _format_selection_exts = {
@@ -583,7 +586,6 @@ class YoutubeDL:
self._playlist_urls = set() self._playlist_urls = set()
self.cache = Cache(self) self.cache = Cache(self)
windows_enable_vt_mode()
stdout = sys.stderr if self.params.get('logtostderr') else sys.stdout stdout = sys.stderr if self.params.get('logtostderr') else sys.stdout
self._out_files = Namespace( self._out_files = Namespace(
out=stdout, out=stdout,
@@ -592,6 +594,12 @@ class YoutubeDL:
console=None if compat_os_name == 'nt' else next( console=None if compat_os_name == 'nt' else next(
filter(supports_terminal_sequences, (sys.stderr, sys.stdout)), None) filter(supports_terminal_sequences, (sys.stderr, sys.stdout)), None)
) )
try:
windows_enable_vt_mode()
except Exception as e:
self.write_debug(f'Failed to enable VT mode: {e}')
self._allow_colors = Namespace(**{ self._allow_colors = Namespace(**{
type_: not self.params.get('no_color') and supports_terminal_sequences(stream) type_: not self.params.get('no_color') and supports_terminal_sequences(stream)
for type_, stream in self._out_files.items_ if type_ != 'console' for type_, stream in self._out_files.items_ if type_ != 'console'
@@ -1067,7 +1075,7 @@ class YoutubeDL:
# correspondingly that is not what we want since we need to keep # correspondingly that is not what we want since we need to keep
# '%%' intact for template dict substitution step. Working around # '%%' intact for template dict substitution step. Working around
# with boundary-alike separator hack. # with boundary-alike separator hack.
sep = ''.join([random.choice(ascii_letters) for _ in range(32)]) sep = ''.join(random.choices(ascii_letters, k=32))
outtmpl = outtmpl.replace('%%', f'%{sep}%').replace('$$', f'${sep}$') outtmpl = outtmpl.replace('%%', f'%{sep}%').replace('$$', f'${sep}$')
# outtmpl should be expand_path'ed before template dict substitution # outtmpl should be expand_path'ed before template dict substitution
@@ -1358,10 +1366,18 @@ class YoutubeDL:
def _match_entry(self, info_dict, incomplete=False, silent=False): def _match_entry(self, info_dict, incomplete=False, silent=False):
"""Returns None if the file should be downloaded""" """Returns None if the file should be downloaded"""
_type = info_dict.get('_type', 'video')
assert incomplete or _type == 'video', 'Only video result can be considered complete'
video_title = info_dict.get('title', info_dict.get('id', 'entry')) video_title = info_dict.get('title', info_dict.get('id', 'entry'))
def check_filter(): def check_filter():
if _type in ('playlist', 'multi_video'):
return
elif _type in ('url', 'url_transparent') and not try_call(
lambda: self.get_info_extractor(info_dict['ie_key']).is_single_video(info_dict['url'])):
return
if 'title' in info_dict: if 'title' in info_dict:
# This can happen when we're just evaluating the playlist # This can happen when we're just evaluating the playlist
title = info_dict['title'] title = info_dict['title']
@@ -1373,6 +1389,7 @@ class YoutubeDL:
if rejecttitle: if rejecttitle:
if re.search(rejecttitle, title, re.IGNORECASE): if re.search(rejecttitle, title, re.IGNORECASE):
return '"' + title + '" title matched reject pattern "' + rejecttitle + '"' return '"' + title + '" title matched reject pattern "' + rejecttitle + '"'
date = info_dict.get('upload_date') date = info_dict.get('upload_date')
if date is not None: if date is not None:
dateRange = self.params.get('daterange', DateRange()) dateRange = self.params.get('daterange', DateRange())
@@ -1616,8 +1633,8 @@ class YoutubeDL:
if result_type in ('url', 'url_transparent'): if result_type in ('url', 'url_transparent'):
ie_result['url'] = sanitize_url( ie_result['url'] = sanitize_url(
ie_result['url'], scheme='http' if self.params.get('prefer_insecure') else 'https') ie_result['url'], scheme='http' if self.params.get('prefer_insecure') else 'https')
if ie_result.get('original_url'): if ie_result.get('original_url') and not extra_info.get('original_url'):
extra_info.setdefault('original_url', ie_result['original_url']) extra_info = {'original_url': ie_result['original_url'], **extra_info}
extract_flat = self.params.get('extract_flat', False) extract_flat = self.params.get('extract_flat', False)
if ((extract_flat == 'in_playlist' and 'playlist' in extra_info) if ((extract_flat == 'in_playlist' and 'playlist' in extra_info)
@@ -1760,7 +1777,7 @@ class YoutubeDL:
return { return {
**info, **info,
'playlist_index': 0, 'playlist_index': 0,
'__last_playlist_index': max(ie_result['requested_entries'] or (0, 0)), '__last_playlist_index': max(ie_result.get('requested_entries') or (0, 0)),
'extractor': ie_result['extractor'], 'extractor': ie_result['extractor'],
'extractor_key': ie_result['extractor_key'], 'extractor_key': ie_result['extractor_key'],
} }
@@ -1816,7 +1833,7 @@ class YoutubeDL:
elif self.params.get('playlistrandom'): elif self.params.get('playlistrandom'):
random.shuffle(entries) random.shuffle(entries)
self.to_screen(f'[{ie_result["extractor"]}] Playlist {title}: Downloading {n_entries} videos' self.to_screen(f'[{ie_result["extractor"]}] Playlist {title}: Downloading {n_entries} items'
f'{format_field(ie_result, "playlist_count", " of %s")}') f'{format_field(ie_result, "playlist_count", " of %s")}')
keep_resolved_entries = self.params.get('extract_flat') != 'discard' keep_resolved_entries = self.params.get('extract_flat') != 'discard'
@@ -1849,14 +1866,13 @@ class YoutubeDL:
resolved_entries[i] = (playlist_index, NO_DEFAULT) resolved_entries[i] = (playlist_index, NO_DEFAULT)
continue continue
self.to_screen('[download] Downloading video %s of %s' % ( self.to_screen('[download] Downloading item %s of %s' % (
self._format_screen(i + 1, self.Styles.ID), self._format_screen(n_entries, self.Styles.EMPHASIS))) self._format_screen(i + 1, self.Styles.ID), self._format_screen(n_entries, self.Styles.EMPHASIS)))
extra.update({ entry_result = self.__process_iterable_entry(entry, download, collections.ChainMap({
'playlist_index': playlist_index, 'playlist_index': playlist_index,
'playlist_autonumber': i + 1, 'playlist_autonumber': i + 1,
}) }, extra))
entry_result = self.__process_iterable_entry(entry, download, extra)
if not entry_result: if not entry_result:
failures += 1 failures += 1
if failures >= max_failures: if failures >= max_failures:
@@ -1867,8 +1883,11 @@ class YoutubeDL:
resolved_entries[i] = (playlist_index, entry_result) resolved_entries[i] = (playlist_index, entry_result)
# Update with processed data # Update with processed data
ie_result['requested_entries'] = [i for i, e in resolved_entries if e is not NO_DEFAULT]
ie_result['entries'] = [e for _, e in resolved_entries if e is not NO_DEFAULT] ie_result['entries'] = [e for _, e in resolved_entries if e is not NO_DEFAULT]
ie_result['requested_entries'] = [i for i, e in resolved_entries if e is not NO_DEFAULT]
if ie_result['requested_entries'] == try_call(lambda: list(range(1, ie_result['playlist_count'] + 1))):
# Do not set for full playlist
ie_result.pop('requested_entries')
# Write the updated info to json # Write the updated info to json
if _infojson_written is True and self._write_info_json( if _infojson_written is True and self._write_info_json(
@@ -2174,6 +2193,7 @@ class YoutubeDL:
'vcodec': the_only_video.get('vcodec'), 'vcodec': the_only_video.get('vcodec'),
'vbr': the_only_video.get('vbr'), 'vbr': the_only_video.get('vbr'),
'stretched_ratio': the_only_video.get('stretched_ratio'), 'stretched_ratio': the_only_video.get('stretched_ratio'),
'aspect_ratio': the_only_video.get('aspect_ratio'),
}) })
if the_only_audio: if the_only_audio:
@@ -2391,11 +2411,7 @@ class YoutubeDL:
def _fill_common_fields(self, info_dict, final=True): def _fill_common_fields(self, info_dict, final=True):
# TODO: move sanitization here # TODO: move sanitization here
if final: if final:
title = info_dict.get('title', NO_DEFAULT) title = info_dict['fulltitle'] = info_dict.get('title')
if title is NO_DEFAULT:
raise ExtractorError('Missing "title" field in extractor result',
video_id=info_dict['id'], ie=info_dict['extractor'])
info_dict['fulltitle'] = title
if not title: if not title:
if title == '': if title == '':
self.write_debug('Extractor gave empty title. Creating a generic title') self.write_debug('Extractor gave empty title. Creating a generic title')
@@ -2448,6 +2464,11 @@ class YoutubeDL:
if err: if err:
self.report_error(err, tb=False) self.report_error(err, tb=False)
def sort_formats(self, info_dict):
formats = self._get_formats(info_dict)
formats.sort(key=FormatSorter(
self, info_dict.get('_format_sort_fields') or []).calculate_preference)
def process_video_result(self, info_dict, download=True): def process_video_result(self, info_dict, download=True):
assert info_dict.get('_type', 'video') == 'video' assert info_dict.get('_type', 'video') == 'video'
self._num_videos += 1 self._num_videos += 1
@@ -2535,6 +2556,11 @@ class YoutubeDL:
formats = self._get_formats(info_dict) formats = self._get_formats(info_dict)
# Backward compatibility with InfoExtractor._sort_formats
field_preference = (formats or [{}])[0].pop('__sort_fields', None)
if field_preference:
info_dict['_format_sort_fields'] = field_preference
# or None ensures --clean-infojson removes it # or None ensures --clean-infojson removes it
info_dict['_has_drm'] = any(f.get('has_drm') for f in formats) or None info_dict['_has_drm'] = any(f.get('has_drm') for f in formats) or None
if not self.params.get('allow_unplayable_formats'): if not self.params.get('allow_unplayable_formats'):
@@ -2572,22 +2598,43 @@ class YoutubeDL:
if not formats: if not formats:
self.raise_no_formats(info_dict) self.raise_no_formats(info_dict)
formats_dict = {} for format in formats:
# We check that all the formats have the format and format_id fields
for i, format in enumerate(formats):
sanitize_string_field(format, 'format_id') sanitize_string_field(format, 'format_id')
sanitize_numeric_fields(format) sanitize_numeric_fields(format)
format['url'] = sanitize_url(format['url']) format['url'] = sanitize_url(format['url'])
if format.get('ext') is None:
format['ext'] = determine_ext(format['url']).lower()
if format.get('protocol') is None:
format['protocol'] = determine_protocol(format)
if format.get('resolution') is None:
format['resolution'] = self.format_resolution(format, default=None)
if format.get('dynamic_range') is None and format.get('vcodec') != 'none':
format['dynamic_range'] = 'SDR'
if format.get('aspect_ratio') is None:
format['aspect_ratio'] = try_call(lambda: round(format['width'] / format['height'], 2))
if (info_dict.get('duration') and format.get('tbr')
and not format.get('filesize') and not format.get('filesize_approx')):
format['filesize_approx'] = int(info_dict['duration'] * format['tbr'] * (1024 / 8))
format['http_headers'] = self._calc_headers(collections.ChainMap(format, info_dict))
# This is copied to http_headers by the above _calc_headers and can now be removed
if '__x_forwarded_for_ip' in info_dict:
del info_dict['__x_forwarded_for_ip']
self.sort_formats({
'formats': formats,
'_format_sort_fields': info_dict.get('_format_sort_fields')
})
# Sanitize and group by format_id
formats_dict = {}
for i, format in enumerate(formats):
if not format.get('format_id'): if not format.get('format_id'):
format['format_id'] = str(i) format['format_id'] = str(i)
else: else:
# Sanitize format_id from characters used in format selector expression # Sanitize format_id from characters used in format selector expression
format['format_id'] = re.sub(r'[\s,/+\[\]()]', '_', format['format_id']) format['format_id'] = re.sub(r'[\s,/+\[\]()]', '_', format['format_id'])
format_id = format['format_id'] formats_dict.setdefault(format['format_id'], []).append(format)
if format_id not in formats_dict:
formats_dict[format_id] = []
formats_dict[format_id].append(format)
# Make sure all formats have unique format_id # Make sure all formats have unique format_id
common_exts = set(itertools.chain(*self._format_selection_exts.values())) common_exts = set(itertools.chain(*self._format_selection_exts.values()))
@@ -2596,38 +2643,17 @@ class YoutubeDL:
for i, format in enumerate(ambiguous_formats): for i, format in enumerate(ambiguous_formats):
if ambigious_id: if ambigious_id:
format['format_id'] = '%s-%d' % (format_id, i) format['format_id'] = '%s-%d' % (format_id, i)
if format.get('ext') is None:
format['ext'] = determine_ext(format['url']).lower()
# Ensure there is no conflict between id and ext in format selection # Ensure there is no conflict between id and ext in format selection
# See https://github.com/yt-dlp/yt-dlp/issues/1282 # See https://github.com/yt-dlp/yt-dlp/issues/1282
if format['format_id'] != format['ext'] and format['format_id'] in common_exts: if format['format_id'] != format['ext'] and format['format_id'] in common_exts:
format['format_id'] = 'f%s' % format['format_id'] format['format_id'] = 'f%s' % format['format_id']
for i, format in enumerate(formats):
if format.get('format') is None: if format.get('format') is None:
format['format'] = '{id} - {res}{note}'.format( format['format'] = '{id} - {res}{note}'.format(
id=format['format_id'], id=format['format_id'],
res=self.format_resolution(format), res=self.format_resolution(format),
note=format_field(format, 'format_note', ' (%s)'), note=format_field(format, 'format_note', ' (%s)'),
) )
if format.get('protocol') is None:
format['protocol'] = determine_protocol(format)
if format.get('resolution') is None:
format['resolution'] = self.format_resolution(format, default=None)
if format.get('dynamic_range') is None and format.get('vcodec') != 'none':
format['dynamic_range'] = 'SDR'
if (info_dict.get('duration') and format.get('tbr')
and not format.get('filesize') and not format.get('filesize_approx')):
format['filesize_approx'] = int(info_dict['duration'] * format['tbr'] * (1024 / 8))
# Add HTTP headers, so that external programs can use them from the
# json output
full_format_info = info_dict.copy()
full_format_info.update(format)
format['http_headers'] = self._calc_headers(full_format_info)
# Remove private housekeeping stuff
if '__x_forwarded_for_ip' in info_dict:
del info_dict['__x_forwarded_for_ip']
if self.params.get('check_formats') is True: if self.params.get('check_formats') is True:
formats = LazyList(self._check_formats(formats[::-1]), reverse=True) formats = LazyList(self._check_formats(formats[::-1]), reverse=True)
@@ -2784,10 +2810,14 @@ class YoutubeDL:
self.params.get('subtitleslangs'), {'all': all_sub_langs}, use_regex=True) self.params.get('subtitleslangs'), {'all': all_sub_langs}, use_regex=True)
except re.error as e: except re.error as e:
raise ValueError(f'Wrong regex for subtitlelangs: {e.pattern}') raise ValueError(f'Wrong regex for subtitlelangs: {e.pattern}')
elif normal_sub_langs:
requested_langs = ['en'] if 'en' in normal_sub_langs else normal_sub_langs[:1]
else: else:
requested_langs = ['en'] if 'en' in all_sub_langs else all_sub_langs[:1] requested_langs = LazyList(itertools.chain(
['en'] if 'en' in normal_sub_langs else [],
filter(lambda f: f.startswith('en'), normal_sub_langs),
['en'] if 'en' in all_sub_langs else [],
filter(lambda f: f.startswith('en'), all_sub_langs),
normal_sub_langs, all_sub_langs,
))[:1]
if requested_langs: if requested_langs:
self.to_screen(f'[info] {video_id}: Downloading subtitles: {", ".join(requested_langs)}') self.to_screen(f'[info] {video_id}: Downloading subtitles: {", ".join(requested_langs)}')
@@ -2942,14 +2972,22 @@ class YoutubeDL:
if 'format' not in info_dict and 'ext' in info_dict: if 'format' not in info_dict and 'ext' in info_dict:
info_dict['format'] = info_dict['ext'] info_dict['format'] = info_dict['ext']
# This is mostly just for backward compatibility of process_info
# As a side-effect, this allows for format-specific filters
if self._match_entry(info_dict) is not None: if self._match_entry(info_dict) is not None:
info_dict['__write_download_archive'] = 'ignore' info_dict['__write_download_archive'] = 'ignore'
return return
# Does nothing under normal operation - for backward compatibility of process_info # Does nothing under normal operation - for backward compatibility of process_info
self.post_extract(info_dict) self.post_extract(info_dict)
def replace_info_dict(new_info):
nonlocal info_dict
if new_info == info_dict:
return
info_dict.clear()
info_dict.update(new_info)
new_info, _ = self.pre_process(info_dict, 'video')
replace_info_dict(new_info)
self._num_downloads += 1 self._num_downloads += 1
# info_dict['_filename'] needs to be set for backward compatibility # info_dict['_filename'] needs to be set for backward compatibility
@@ -3063,13 +3101,6 @@ class YoutubeDL:
for link_type, should_write in write_links.items()): for link_type, should_write in write_links.items()):
return return
def replace_info_dict(new_info):
nonlocal info_dict
if new_info == info_dict:
return
info_dict.clear()
info_dict.update(new_info)
new_info, files_to_move = self.pre_process(info_dict, 'before_dl', files_to_move) new_info, files_to_move = self.pre_process(info_dict, 'before_dl', files_to_move)
replace_info_dict(new_info) replace_info_dict(new_info)
@@ -3096,7 +3127,7 @@ class YoutubeDL:
fd, success = None, True fd, success = None, True
if info_dict.get('protocol') or info_dict.get('url'): if info_dict.get('protocol') or info_dict.get('url'):
fd = get_suitable_downloader(info_dict, self.params, to_stdout=temp_filename == '-') fd = get_suitable_downloader(info_dict, self.params, to_stdout=temp_filename == '-')
if fd is not FFmpegFD and ( if fd is not FFmpegFD and 'no-direct-merge' not in self.params['compat_opts'] and (
info_dict.get('section_start') or info_dict.get('section_end')): info_dict.get('section_start') or info_dict.get('section_end')):
msg = ('This format cannot be partially downloaded' if FFmpegFD.available() msg = ('This format cannot be partially downloaded' if FFmpegFD.available()
else 'You have requested downloading the video partially, but ffmpeg is not installed') else 'You have requested downloading the video partially, but ffmpeg is not installed')
@@ -3361,6 +3392,7 @@ class YoutubeDL:
reject = lambda k, v: v is None or k.startswith('__') or k in { reject = lambda k, v: v is None or k.startswith('__') or k in {
'requested_downloads', 'requested_formats', 'requested_subtitles', 'requested_entries', 'requested_downloads', 'requested_formats', 'requested_subtitles', 'requested_entries',
'entries', 'filepath', '_filename', 'infojson_filename', 'original_url', 'playlist_autonumber', 'entries', 'filepath', '_filename', 'infojson_filename', 'original_url', 'playlist_autonumber',
'_format_sort_fields',
} }
else: else:
reject = lambda k, v: False reject = lambda k, v: False
@@ -3430,6 +3462,7 @@ class YoutubeDL:
return infodict return infodict
def run_all_pps(self, key, info, *, additional_pps=None): def run_all_pps(self, key, info, *, additional_pps=None):
if key != 'video':
self._forceprint(key, info) self._forceprint(key, info)
for pp in (additional_pps or []) + self._pps[key]: for pp in (additional_pps or []) + self._pps[key]:
info = self.run_pp(pp, info) info = self.run_pp(pp, info)
@@ -3632,6 +3665,7 @@ class YoutubeDL:
format_field(f, 'asr', '\t%s', func=format_decimal_suffix), format_field(f, 'asr', '\t%s', func=format_decimal_suffix),
join_nonempty( join_nonempty(
self._format_out('UNSUPPORTED', 'light red') if f.get('ext') in ('f4f', 'f4m') else None, self._format_out('UNSUPPORTED', 'light red') if f.get('ext') in ('f4f', 'f4m') else None,
self._format_out('DRM', 'light red') if f.get('has_drm') else None,
format_field(f, 'language', '[%s]'), format_field(f, 'language', '[%s]'),
join_nonempty(format_field(f, 'format_note'), join_nonempty(format_field(f, 'format_note'),
format_field(f, 'container', ignore=(None, f.get('ext'))), format_field(f, 'container', ignore=(None, f.get('ext'))),
@@ -3699,7 +3733,10 @@ class YoutubeDL:
# These imports can be slow. So import them only as needed # These imports can be slow. So import them only as needed
from .extractor.extractors import _LAZY_LOADER from .extractor.extractors import _LAZY_LOADER
from .extractor.extractors import _PLUGIN_CLASSES as plugin_extractors from .extractor.extractors import (
_PLUGIN_CLASSES as plugin_ies,
_PLUGIN_OVERRIDES as plugin_ie_overrides
)
def get_encoding(stream): def get_encoding(stream):
ret = str(getattr(stream, 'encoding', 'missing (%s)' % type(stream).__name__)) ret = str(getattr(stream, 'encoding', 'missing (%s)' % type(stream).__name__))
@@ -3728,12 +3765,13 @@ class YoutubeDL:
source = detect_variant() source = detect_variant()
if VARIANT not in (None, 'pip'): if VARIANT not in (None, 'pip'):
source += '*' source += '*'
klass = type(self)
write_debug(join_nonempty( write_debug(join_nonempty(
f'{"yt-dlp" if REPOSITORY == "yt-dlp/yt-dlp" else REPOSITORY} version', f'{"yt-dlp" if REPOSITORY == "yt-dlp/yt-dlp" else REPOSITORY} version',
__version__, __version__,
f'[{RELEASE_GIT_HEAD}]' if RELEASE_GIT_HEAD else '', f'[{RELEASE_GIT_HEAD}]' if RELEASE_GIT_HEAD else '',
'' if source == 'unknown' else f'({source})', '' if source == 'unknown' else f'({source})',
'' if _IN_CLI else 'API', '' if _IN_CLI else 'API' if klass == YoutubeDL else f'API:{self.__module__}.{klass.__qualname__}',
delim=' ')) delim=' '))
if not _IN_CLI: if not _IN_CLI:
@@ -3744,10 +3782,6 @@ class YoutubeDL:
write_debug('Lazy loading extractors is forcibly disabled') write_debug('Lazy loading extractors is forcibly disabled')
else: else:
write_debug('Lazy loading extractors is disabled') write_debug('Lazy loading extractors is disabled')
if plugin_extractors or plugin_postprocessors:
write_debug('Plugins: %s' % [
'%s%s' % (klass.__name__, '' if klass.__name__ == name else f' as {name}')
for name, klass in itertools.chain(plugin_extractors.items(), plugin_postprocessors.items())])
if self.params['compat_opts']: if self.params['compat_opts']:
write_debug('Compatibility options: %s' % ', '.join(self.params['compat_opts'])) write_debug('Compatibility options: %s' % ', '.join(self.params['compat_opts']))
@@ -3781,6 +3815,21 @@ class YoutubeDL:
proxy_map.update(handler.proxies) proxy_map.update(handler.proxies)
write_debug(f'Proxy map: {proxy_map}') write_debug(f'Proxy map: {proxy_map}')
for plugin_type, plugins in {'Extractor': plugin_ies, 'Post-Processor': plugin_pps}.items():
display_list = ['%s%s' % (
klass.__name__, '' if klass.__name__ == name else f' as {name}')
for name, klass in plugins.items()]
if plugin_type == 'Extractor':
display_list.extend(f'{plugins[-1].IE_NAME.partition("+")[2]} ({parent.__name__})'
for parent, plugins in plugin_ie_overrides.items())
if not display_list:
continue
write_debug(f'{plugin_type} Plugins: {", ".join(sorted(display_list))}')
plugin_dirs = plugin_directories()
if plugin_dirs:
write_debug(f'Plugin directories: {plugin_dirs}')
# Not implemented # Not implemented
if False and self.params.get('call_home'): if False and self.params.get('call_home'):
ipaddr = self.urlopen('https://yt-dl.org/ip').read().decode() ipaddr = self.urlopen('https://yt-dl.org/ip').read().decode()
@@ -3830,8 +3879,11 @@ class YoutubeDL:
# https://github.com/ytdl-org/youtube-dl/issues/8227) # https://github.com/ytdl-org/youtube-dl/issues/8227)
file_handler = urllib.request.FileHandler() file_handler = urllib.request.FileHandler()
if not self.params.get('enable_file_urls'):
def file_open(*args, **kwargs): def file_open(*args, **kwargs):
raise urllib.error.URLError('file:// scheme is explicitly disabled in yt-dlp for security reasons') raise urllib.error.URLError(
'file:// URLs are explicitly disabled in yt-dlp for security reasons. '
'Use --enable-file-urls to enable at your own risk.')
file_handler.file_open = file_open file_handler.file_open = file_open
opener = urllib.request.build_opener( opener = urllib.request.build_opener(
@@ -3894,7 +3946,7 @@ class YoutubeDL:
elif not self.params.get('overwrites', True) and os.path.exists(descfn): elif not self.params.get('overwrites', True) and os.path.exists(descfn):
self.to_screen(f'[info] {label.title()} description is already present') self.to_screen(f'[info] {label.title()} description is already present')
elif ie_result.get('description') is None: elif ie_result.get('description') is None:
self.report_warning(f'There\'s no {label} description to write') self.to_screen(f'[info] There\'s no {label} description to write')
return False return False
else: else:
try: try:
@@ -3910,15 +3962,18 @@ class YoutubeDL:
''' Write subtitles to file and return list of (sub_filename, final_sub_filename); or None if error''' ''' Write subtitles to file and return list of (sub_filename, final_sub_filename); or None if error'''
ret = [] ret = []
subtitles = info_dict.get('requested_subtitles') subtitles = info_dict.get('requested_subtitles')
if not subtitles or not (self.params.get('writesubtitles') or self.params.get('writeautomaticsub')): if not (self.params.get('writesubtitles') or self.params.get('writeautomaticsub')):
# subtitles download errors are already managed as troubles in relevant IE # subtitles download errors are already managed as troubles in relevant IE
# that way it will silently go on when used with unsupporting IE # that way it will silently go on when used with unsupporting IE
return ret return ret
elif not subtitles:
self.to_screen('[info] There\'s no subtitles for the requested languages')
return ret
sub_filename_base = self.prepare_filename(info_dict, 'subtitle') sub_filename_base = self.prepare_filename(info_dict, 'subtitle')
if not sub_filename_base: if not sub_filename_base:
self.to_screen('[info] Skipping writing video subtitles') self.to_screen('[info] Skipping writing video subtitles')
return ret return ret
for sub_lang, sub_info in subtitles.items(): for sub_lang, sub_info in subtitles.items():
sub_format = sub_info['ext'] sub_format = sub_info['ext']
sub_filename = subtitles_filename(filename, sub_lang, sub_format, info_dict.get('ext')) sub_filename = subtitles_filename(filename, sub_lang, sub_format, info_dict.get('ext'))
@@ -3965,6 +4020,9 @@ class YoutubeDL:
thumbnails, ret = [], [] thumbnails, ret = [], []
if write_all or self.params.get('writethumbnail', False): if write_all or self.params.get('writethumbnail', False):
thumbnails = info_dict.get('thumbnails') or [] thumbnails = info_dict.get('thumbnails') or []
if not thumbnails:
self.to_screen(f'[info] There\'s no {label} thumbnails to download')
return ret
multiple = write_all and len(thumbnails) > 1 multiple = write_all and len(thumbnails) > 1
if thumb_filename_base is None: if thumb_filename_base is None:
+29 -30
View File
@@ -16,11 +16,9 @@ import sys
from .compat import compat_shlex_quote from .compat import compat_shlex_quote
from .cookies import SUPPORTED_BROWSERS, SUPPORTED_KEYRINGS from .cookies import SUPPORTED_BROWSERS, SUPPORTED_KEYRINGS
from .downloader import FileDownloader
from .downloader.external import get_external_downloader from .downloader.external import get_external_downloader
from .extractor import list_extractor_classes from .extractor import list_extractor_classes
from .extractor.adobepass import MSO_INFO from .extractor.adobepass import MSO_INFO
from .extractor.common import InfoExtractor
from .options import parseOpts from .options import parseOpts
from .postprocessor import ( from .postprocessor import (
FFmpegExtractAudioPP, FFmpegExtractAudioPP,
@@ -40,6 +38,7 @@ from .utils import (
DateRange, DateRange,
DownloadCancelled, DownloadCancelled,
DownloadError, DownloadError,
FormatSorter,
GeoUtils, GeoUtils,
PlaylistEntries, PlaylistEntries,
SameFileError, SameFileError,
@@ -50,6 +49,7 @@ from .utils import (
format_field, format_field,
int_or_none, int_or_none,
match_filter_func, match_filter_func,
parse_bytes,
parse_duration, parse_duration,
preferredencoding, preferredencoding,
read_batch_urls, read_batch_urls,
@@ -91,12 +91,11 @@ def get_urls(urls, batchfile, verbose):
def print_extractor_information(opts, urls): def print_extractor_information(opts, urls):
# Importing GenericIE is currently slow since it imports other extractors
# TODO: Move this back to module level after generalization of embed detection
from .extractor.generic import GenericIE
out = '' out = ''
if opts.list_extractors: if opts.list_extractors:
# Importing GenericIE is currently slow since it imports YoutubeIE
from .extractor.generic import GenericIE
urls = dict.fromkeys(urls, False) urls = dict.fromkeys(urls, False)
for ie in list_extractor_classes(opts.age_limit): for ie in list_extractor_classes(opts.age_limit):
out += ie.IE_NAME + (' (CURRENTLY BROKEN)' if not ie.working() else '') + '\n' out += ie.IE_NAME + (' (CURRENTLY BROKEN)' if not ie.working() else '') + '\n'
@@ -152,7 +151,7 @@ def set_compat_opts(opts):
else: else:
opts.embed_infojson = False opts.embed_infojson = False
if 'format-sort' in opts.compat_opts: if 'format-sort' in opts.compat_opts:
opts.format_sort.extend(InfoExtractor.FormatSort.ytdl_default) opts.format_sort.extend(FormatSorter.ytdl_default)
_video_multistreams_set = set_default_compat('multistreams', 'allow_multiple_video_streams', False, remove_compat=False) _video_multistreams_set = set_default_compat('multistreams', 'allow_multiple_video_streams', False, remove_compat=False)
_audio_multistreams_set = set_default_compat('multistreams', 'allow_multiple_audio_streams', False, remove_compat=False) _audio_multistreams_set = set_default_compat('multistreams', 'allow_multiple_audio_streams', False, remove_compat=False)
if _video_multistreams_set is False and _audio_multistreams_set is False: if _video_multistreams_set is False and _audio_multistreams_set is False:
@@ -227,7 +226,7 @@ def validate_options(opts):
# Format sort # Format sort
for f in opts.format_sort: for f in opts.format_sort:
validate_regex('format sorting', f, InfoExtractor.FormatSort.regex) validate_regex('format sorting', f, FormatSorter.regex)
# Postprocessor formats # Postprocessor formats
validate_regex('merge output format', opts.merge_output_format, validate_regex('merge output format', opts.merge_output_format,
@@ -281,19 +280,19 @@ def validate_options(opts):
raise ValueError(f'invalid {key} retry sleep expression {expr!r}') raise ValueError(f'invalid {key} retry sleep expression {expr!r}')
# Bytes # Bytes
def parse_bytes(name, value): def validate_bytes(name, value):
if value is None: if value is None:
return None return None
numeric_limit = FileDownloader.parse_bytes(value) numeric_limit = parse_bytes(value)
validate(numeric_limit is not None, 'rate limit', value) validate(numeric_limit is not None, 'rate limit', value)
return numeric_limit return numeric_limit
opts.ratelimit = parse_bytes('rate limit', opts.ratelimit) opts.ratelimit = validate_bytes('rate limit', opts.ratelimit)
opts.throttledratelimit = parse_bytes('throttled rate limit', opts.throttledratelimit) opts.throttledratelimit = validate_bytes('throttled rate limit', opts.throttledratelimit)
opts.min_filesize = parse_bytes('min filesize', opts.min_filesize) opts.min_filesize = validate_bytes('min filesize', opts.min_filesize)
opts.max_filesize = parse_bytes('max filesize', opts.max_filesize) opts.max_filesize = validate_bytes('max filesize', opts.max_filesize)
opts.buffersize = parse_bytes('buffer size', opts.buffersize) opts.buffersize = validate_bytes('buffer size', opts.buffersize)
opts.http_chunk_size = parse_bytes('http chunk size', opts.http_chunk_size) opts.http_chunk_size = validate_bytes('http chunk size', opts.http_chunk_size)
# Output templates # Output templates
def validate_outtmpl(tmpl, msg): def validate_outtmpl(tmpl, msg):
@@ -319,10 +318,6 @@ def validate_options(opts):
if outtmpl_default == '': if outtmpl_default == '':
opts.skip_download = None opts.skip_download = None
del opts.outtmpl['default'] del opts.outtmpl['default']
if outtmpl_default and not os.path.splitext(outtmpl_default)[1] and opts.extractaudio:
raise ValueError(
'Cannot download a video and extract audio into the same file! '
f'Use "{outtmpl_default}.%(ext)s" instead of "{outtmpl_default}" as the output template')
def parse_chapters(name, value): def parse_chapters(name, value):
chapters, ranges = [], [] chapters, ranges = [], []
@@ -333,7 +328,7 @@ def validate_options(opts):
mobj = range_ != '-' and re.fullmatch(r'([^-]+)?\s*-\s*([^-]+)?', range_) mobj = range_ != '-' and re.fullmatch(r'([^-]+)?\s*-\s*([^-]+)?', range_)
dur = mobj and (parse_timestamp(mobj.group(1) or '0'), parse_timestamp(mobj.group(2) or 'inf')) dur = mobj and (parse_timestamp(mobj.group(1) or '0'), parse_timestamp(mobj.group(2) or 'inf'))
if None in (dur or [None]): if None in (dur or [None]):
raise ValueError(f'invalid {name} time range "{regex}". Must be of the form *start-end') raise ValueError(f'invalid {name} time range "{regex}". Must be of the form "*start-end"')
ranges.append(dur) ranges.append(dur)
continue continue
try: try:
@@ -351,7 +346,7 @@ def validate_options(opts):
mobj = re.fullmatch(r'''(?x) mobj = re.fullmatch(r'''(?x)
(?P<name>[^+:]+) (?P<name>[^+:]+)
(?:\s*\+\s*(?P<keyring>[^:]+))? (?:\s*\+\s*(?P<keyring>[^:]+))?
(?:\s*:\s*(?P<profile>.+?))? (?:\s*:\s*(?!:)(?P<profile>.+?))?
(?:\s*::\s*(?P<container>.+))? (?:\s*::\s*(?P<container>.+))?
''', opts.cookiesfrombrowser) ''', opts.cookiesfrombrowser)
if mobj is None: if mobj is None:
@@ -387,10 +382,12 @@ def validate_options(opts):
raise ValueError(f'{cmd} is invalid; {err}') raise ValueError(f'{cmd} is invalid; {err}')
yield action yield action
parse_metadata = opts.parse_metadata or []
if opts.metafromtitle is not None: if opts.metafromtitle is not None:
parse_metadata.append('title:%s' % opts.metafromtitle) opts.parse_metadata.setdefault('pre_process', []).append('title:%s' % opts.metafromtitle)
opts.parse_metadata = list(itertools.chain(*map(metadataparser_actions, parse_metadata))) opts.parse_metadata = {
k: list(itertools.chain(*map(metadataparser_actions, v)))
for k, v in opts.parse_metadata.items()
}
# Other options # Other options
if opts.playlist_items is not None: if opts.playlist_items is not None:
@@ -562,11 +559,11 @@ def validate_options(opts):
def get_postprocessors(opts): def get_postprocessors(opts):
yield from opts.add_postprocessors yield from opts.add_postprocessors
if opts.parse_metadata: for when, actions in opts.parse_metadata.items():
yield { yield {
'key': 'MetadataParser', 'key': 'MetadataParser',
'actions': opts.parse_metadata, 'actions': actions,
'when': 'pre_process' 'when': when
} }
sponsorblock_query = opts.sponsorblock_mark | opts.sponsorblock_remove sponsorblock_query = opts.sponsorblock_mark | opts.sponsorblock_remove
if sponsorblock_query: if sponsorblock_query:
@@ -702,11 +699,12 @@ def parse_options(argv=None):
postprocessors = list(get_postprocessors(opts)) postprocessors = list(get_postprocessors(opts))
print_only = bool(opts.forceprint) and all(k not in opts.forceprint for k in POSTPROCESS_WHEN[2:]) print_only = bool(opts.forceprint) and all(k not in opts.forceprint for k in POSTPROCESS_WHEN[3:])
any_getting = any(getattr(opts, k) for k in ( any_getting = any(getattr(opts, k) for k in (
'dumpjson', 'dump_single_json', 'getdescription', 'getduration', 'getfilename', 'dumpjson', 'dump_single_json', 'getdescription', 'getduration', 'getfilename',
'getformat', 'getid', 'getthumbnail', 'gettitle', 'geturl' 'getformat', 'getid', 'getthumbnail', 'gettitle', 'geturl'
)) ))
opts.quiet = opts.quiet or any_getting or opts.print_json or bool(opts.forceprint)
playlist_pps = [pp for pp in postprocessors if pp.get('when') == 'playlist'] playlist_pps = [pp for pp in postprocessors if pp.get('when') == 'playlist']
write_playlist_infojson = (opts.writeinfojson and not opts.clean_infojson write_playlist_infojson = (opts.writeinfojson and not opts.clean_infojson
@@ -742,7 +740,7 @@ def parse_options(argv=None):
'client_certificate': opts.client_certificate, 'client_certificate': opts.client_certificate,
'client_certificate_key': opts.client_certificate_key, 'client_certificate_key': opts.client_certificate_key,
'client_certificate_password': opts.client_certificate_password, 'client_certificate_password': opts.client_certificate_password,
'quiet': opts.quiet or any_getting or opts.print_json or bool(opts.forceprint), 'quiet': opts.quiet,
'no_warnings': opts.no_warnings, 'no_warnings': opts.no_warnings,
'forceurl': opts.geturl, 'forceurl': opts.geturl,
'forcetitle': opts.gettitle, 'forcetitle': opts.gettitle,
@@ -854,6 +852,7 @@ def parse_options(argv=None):
'legacyserverconnect': opts.legacy_server_connect, 'legacyserverconnect': opts.legacy_server_connect,
'nocheckcertificate': opts.no_check_certificate, 'nocheckcertificate': opts.no_check_certificate,
'prefer_insecure': opts.prefer_insecure, 'prefer_insecure': opts.prefer_insecure,
'enable_file_urls': opts.enable_file_urls,
'http_headers': opts.headers, 'http_headers': opts.headers,
'proxy': opts.proxy, 'proxy': opts.proxy,
'socket_timeout': opts.socket_timeout, 'socket_timeout': opts.socket_timeout,
+5
View File
@@ -0,0 +1,5 @@
import os
def get_hook_dirs():
return [os.path.dirname(__file__)]
+57
View File
@@ -0,0 +1,57 @@
import ast
import os
import sys
from pathlib import Path
from PyInstaller.utils.hooks import collect_submodules
def find_attribute_accesses(node, name, path=()):
if isinstance(node, ast.Attribute):
path = [*path, node.attr]
if isinstance(node.value, ast.Name) and node.value.id == name:
yield path[::-1]
for child in ast.iter_child_nodes(node):
yield from find_attribute_accesses(child, name, path)
def collect_used_submodules(name, level):
for dirpath, _, filenames in os.walk(Path(__file__).parent.parent):
for filename in filenames:
if not filename.endswith('.py'):
continue
with open(Path(dirpath) / filename, encoding='utf8') as f:
for submodule in find_attribute_accesses(ast.parse(f.read()), name):
yield '.'.join(submodule[:level])
def pycryptodome_module():
try:
import Cryptodome # noqa: F401
except ImportError:
try:
import Crypto # noqa: F401
print('WARNING: Using Crypto since Cryptodome is not available. '
'Install with: pip install pycryptodomex', file=sys.stderr)
return 'Crypto'
except ImportError:
pass
return 'Cryptodome'
def get_hidden_imports():
yield 'yt_dlp.compat._legacy'
yield from collect_submodules('websockets')
crypto = pycryptodome_module()
for sm in set(collect_used_submodules('Cryptodome', 2)):
yield f'{crypto}.{sm}'
# These are auto-detected, but explicitly add them just in case
yield from ('mutagen', 'brotli', 'certifi')
hiddenimports = list(get_hidden_imports())
print(f'Adding imports: {hiddenimports}')
excludedimports = ['youtube_dl', 'youtube_dlc', 'test', 'ytdlp_plugins', 'devscripts']
+4 -4
View File
@@ -2,17 +2,17 @@ import base64
from math import ceil from math import ceil
from .compat import compat_ord from .compat import compat_ord
from .dependencies import Cryptodome_AES from .dependencies import Cryptodome
from .utils import bytes_to_intlist, intlist_to_bytes from .utils import bytes_to_intlist, intlist_to_bytes
if Cryptodome_AES: if Cryptodome:
def aes_cbc_decrypt_bytes(data, key, iv): def aes_cbc_decrypt_bytes(data, key, iv):
""" Decrypt bytes with AES-CBC using pycryptodome """ """ Decrypt bytes with AES-CBC using pycryptodome """
return Cryptodome_AES.new(key, Cryptodome_AES.MODE_CBC, iv).decrypt(data) return Cryptodome.Cipher.AES.new(key, Cryptodome.Cipher.AES.MODE_CBC, iv).decrypt(data)
def aes_gcm_decrypt_and_verify_bytes(data, key, tag, nonce): def aes_gcm_decrypt_and_verify_bytes(data, key, tag, nonce):
""" Decrypt bytes with AES-GCM using pycryptodome """ """ Decrypt bytes with AES-GCM using pycryptodome """
return Cryptodome_AES.new(key, Cryptodome_AES.MODE_GCM, nonce).decrypt_and_verify(data, tag) return Cryptodome.Cipher.AES.new(key, Cryptodome.Cipher.AES.MODE_GCM, nonce).decrypt_and_verify(data, tag)
else: else:
def aes_cbc_decrypt_bytes(data, key, iv): def aes_cbc_decrypt_bytes(data, key, iv):
+5 -11
View File
@@ -1,10 +1,10 @@
import contextlib import contextlib
import errno
import json import json
import os import os
import re import re
import shutil import shutil
import traceback import traceback
import urllib.parse
from .utils import expand_path, traverse_obj, version_tuple, write_json_file from .utils import expand_path, traverse_obj, version_tuple, write_json_file
from .version import __version__ from .version import __version__
@@ -22,11 +22,9 @@ class Cache:
return expand_path(res) return expand_path(res)
def _get_cache_fn(self, section, key, dtype): def _get_cache_fn(self, section, key, dtype):
assert re.match(r'^[a-zA-Z0-9_.-]+$', section), \ assert re.match(r'^[\w.-]+$', section), f'invalid section {section!r}'
'invalid section %r' % section key = urllib.parse.quote(key, safe='').replace('%', ',') # encode non-ascii characters
assert re.match(r'^[a-zA-Z0-9_.-]+$', key), 'invalid key %r' % key return os.path.join(self._get_root_dir(), section, f'{key}.{dtype}')
return os.path.join(
self._get_root_dir(), section, f'{key}.{dtype}')
@property @property
def enabled(self): def enabled(self):
@@ -40,11 +38,7 @@ class Cache:
fn = self._get_cache_fn(section, key, dtype) fn = self._get_cache_fn(section, key, dtype)
try: try:
try: os.makedirs(os.path.dirname(fn), exist_ok=True)
os.makedirs(os.path.dirname(fn))
except OSError as ose:
if ose.errno != errno.EEXIST:
raise
self._ydl.write_debug(f'Saving {section}.{key} to cache') self._ydl.write_debug(f'Saving {section}.{key} to cache')
write_json_file({'yt-dlp_version': __version__, 'data': data}, fn) write_json_file({'yt-dlp_version': __version__, 'data': data}, fn)
except Exception: except Exception:
+1 -7
View File
@@ -8,7 +8,7 @@ from .compat_utils import passthrough_module
# XXX: Implement this the same way as other DeprecationWarnings without circular import # XXX: Implement this the same way as other DeprecationWarnings without circular import
passthrough_module(__name__, '._legacy', callback=lambda attr: warnings.warn( passthrough_module(__name__, '._legacy', callback=lambda attr: warnings.warn(
DeprecationWarning(f'{__name__}.{attr} is deprecated'), stacklevel=3)) DeprecationWarning(f'{__name__}.{attr} is deprecated'), stacklevel=5))
# HTMLParseError has been deprecated in Python 3.3 and removed in # HTMLParseError has been deprecated in Python 3.3 and removed in
@@ -70,9 +70,3 @@ if compat_os_name in ('nt', 'ce'):
return userhome + path[i:] return userhome + path[i:]
else: else:
compat_expanduser = os.path.expanduser compat_expanduser = os.path.expanduser
# NB: Add modules that are imported dynamically here so that PyInstaller can find them
# See https://github.com/pyinstaller/pyinstaller-hooks-contrib/issues/438
if False:
from . import _legacy # noqa: F401
+20 -10
View File
@@ -1,5 +1,6 @@
""" Do not use! """ """ Do not use! """
import base64
import collections import collections
import ctypes import ctypes
import getpass import getpass
@@ -29,6 +30,7 @@ from asyncio import run as compat_asyncio_run # noqa: F401
from re import Pattern as compat_Pattern # noqa: F401 from re import Pattern as compat_Pattern # noqa: F401
from re import match as compat_Match # noqa: F401 from re import match as compat_Match # noqa: F401
from . import compat_expanduser, compat_HTMLParseError, compat_realpath
from .compat_utils import passthrough_module from .compat_utils import passthrough_module
from ..dependencies import Cryptodome_AES as compat_pycrypto_AES # noqa: F401 from ..dependencies import Cryptodome_AES as compat_pycrypto_AES # noqa: F401
from ..dependencies import brotli as compat_brotli # noqa: F401 from ..dependencies import brotli as compat_brotli # noqa: F401
@@ -47,23 +49,25 @@ def compat_setenv(key, value, env=os.environ):
env[key] = value env[key] = value
compat_base64_b64decode = base64.b64decode
compat_basestring = str compat_basestring = str
compat_casefold = str.casefold compat_casefold = str.casefold
compat_chr = chr compat_chr = chr
compat_collections_abc = collections.abc compat_collections_abc = collections.abc
compat_cookiejar = http.cookiejar compat_cookiejar = compat_http_cookiejar = http.cookiejar
compat_cookiejar_Cookie = http.cookiejar.Cookie compat_cookiejar_Cookie = compat_http_cookiejar_Cookie = http.cookiejar.Cookie
compat_cookies = http.cookies compat_cookies = compat_http_cookies = http.cookies
compat_cookies_SimpleCookie = http.cookies.SimpleCookie compat_cookies_SimpleCookie = compat_http_cookies_SimpleCookie = http.cookies.SimpleCookie
compat_etree_Element = etree.Element compat_etree_Element = compat_xml_etree_ElementTree_Element = etree.Element
compat_etree_register_namespace = etree.register_namespace compat_etree_register_namespace = compat_xml_etree_register_namespace = etree.register_namespace
compat_filter = filter compat_filter = filter
compat_get_terminal_size = shutil.get_terminal_size compat_get_terminal_size = shutil.get_terminal_size
compat_getenv = os.getenv compat_getenv = os.getenv
compat_getpass = getpass.getpass compat_getpass = compat_getpass_getpass = getpass.getpass
compat_html_entities = html.entities compat_html_entities = html.entities
compat_html_entities_html5 = html.entities.html5 compat_html_entities_html5 = html.entities.html5
compat_HTMLParser = html.parser.HTMLParser compat_html_parser_HTMLParseError = compat_HTMLParseError
compat_HTMLParser = compat_html_parser_HTMLParser = html.parser.HTMLParser
compat_http_client = http.client compat_http_client = http.client
compat_http_server = http.server compat_http_server = http.server
compat_input = input compat_input = input
@@ -72,6 +76,8 @@ compat_itertools_count = itertools.count
compat_kwargs = lambda kwargs: kwargs compat_kwargs = lambda kwargs: kwargs
compat_map = map compat_map = map
compat_numeric_types = (int, float, complex) compat_numeric_types = (int, float, complex)
compat_os_path_expanduser = compat_expanduser
compat_os_path_realpath = compat_realpath
compat_print = print compat_print = print
compat_shlex_split = shlex.split compat_shlex_split = shlex.split
compat_socket_create_connection = socket.create_connection compat_socket_create_connection = socket.create_connection
@@ -81,7 +87,9 @@ compat_struct_unpack = struct.unpack
compat_subprocess_get_DEVNULL = lambda: DEVNULL compat_subprocess_get_DEVNULL = lambda: DEVNULL
compat_tokenize_tokenize = tokenize.tokenize compat_tokenize_tokenize = tokenize.tokenize
compat_urllib_error = urllib.error compat_urllib_error = urllib.error
compat_urllib_HTTPError = urllib.error.HTTPError
compat_urllib_parse = urllib.parse compat_urllib_parse = urllib.parse
compat_urllib_parse_parse_qs = urllib.parse.parse_qs
compat_urllib_parse_quote = urllib.parse.quote compat_urllib_parse_quote = urllib.parse.quote
compat_urllib_parse_quote_plus = urllib.parse.quote_plus compat_urllib_parse_quote_plus = urllib.parse.quote_plus
compat_urllib_parse_unquote_plus = urllib.parse.unquote_plus compat_urllib_parse_unquote_plus = urllib.parse.unquote_plus
@@ -90,8 +98,10 @@ compat_urllib_parse_urlunparse = urllib.parse.urlunparse
compat_urllib_request = urllib.request compat_urllib_request = urllib.request
compat_urllib_request_DataHandler = urllib.request.DataHandler compat_urllib_request_DataHandler = urllib.request.DataHandler
compat_urllib_response = urllib.response compat_urllib_response = urllib.response
compat_urlretrieve = urllib.request.urlretrieve compat_urlretrieve = compat_urllib_request_urlretrieve = urllib.request.urlretrieve
compat_xml_parse_error = etree.ParseError compat_xml_parse_error = compat_xml_etree_ElementTree_ParseError = etree.ParseError
compat_xpath = lambda xpath: xpath compat_xpath = lambda xpath: xpath
compat_zip = zip compat_zip = zip
workaround_optparse_bug9161 = lambda: None workaround_optparse_bug9161 = lambda: None
legacy = []
+51 -38
View File
@@ -1,5 +1,6 @@
import collections import collections
import contextlib import contextlib
import functools
import importlib import importlib
import sys import sys
import types import types
@@ -10,61 +11,73 @@ _Package = collections.namedtuple('Package', ('name', 'version'))
def get_package_info(module): def get_package_info(module):
parent = module.__name__.split('.')[0] return _Package(
parent_module = None name=getattr(module, '_yt_dlp__identifier', module.__name__),
with contextlib.suppress(ImportError): version=str(next(filter(None, (
parent_module = importlib.import_module(parent) getattr(module, attr, None)
for attr in ('__version__', 'version_string', 'version')
for attr in ('__version__', 'version_string', 'version'): )), None)))
version = getattr(parent_module, attr, None)
if version is not None:
break
return _Package(getattr(module, '_yt_dlp__identifier', parent), str(version))
def _is_package(module): def _is_package(module):
return '__path__' in vars(module)
def _is_dunder(name):
return name.startswith('__') and name.endswith('__')
class EnhancedModule(types.ModuleType):
def __bool__(self):
return vars(self).get('__bool__', lambda: True)()
def __getattribute__(self, attr):
try: try:
module.__getattribute__('__path__') ret = super().__getattribute__(attr)
except AttributeError: except AttributeError:
return False if _is_dunder(attr):
return True raise
getter = getattr(self, '__getattr__', None)
if not getter:
raise
ret = getter(attr)
return ret.fget() if isinstance(ret, property) else ret
def passthrough_module(parent, child, allowed_attributes=None, *, callback=lambda _: None): def passthrough_module(parent, child, allowed_attributes=(..., ), *, callback=lambda _: None):
parent_module = importlib.import_module(parent) """Passthrough parent module into a child module, creating the parent if necessary"""
child_module = None # Import child module only as needed def __getattr__(attr):
if _is_package(parent):
class PassthroughModule(types.ModuleType):
def __getattr__(self, attr):
if _is_package(parent_module):
with contextlib.suppress(ImportError): with contextlib.suppress(ImportError):
return importlib.import_module(f'.{attr}', parent) return importlib.import_module(f'.{attr}', parent.__name__)
ret = self.__from_child(attr) ret = from_child(attr)
if ret is _NO_ATTRIBUTE: if ret is _NO_ATTRIBUTE:
raise AttributeError(f'module {parent} has no attribute {attr}') raise AttributeError(f'module {parent.__name__} has no attribute {attr}')
callback(attr) callback(attr)
return ret return ret
def __from_child(self, attr): @functools.lru_cache(maxsize=None)
if allowed_attributes is None: def from_child(attr):
if attr.startswith('__') and attr.endswith('__'): nonlocal child
return _NO_ATTRIBUTE if attr not in allowed_attributes:
elif attr not in allowed_attributes: if ... not in allowed_attributes or _is_dunder(attr):
return _NO_ATTRIBUTE return _NO_ATTRIBUTE
nonlocal child_module if isinstance(child, str):
child_module = child_module or importlib.import_module(child, parent) child = importlib.import_module(child, parent.__name__)
if _is_package(child):
with contextlib.suppress(ImportError):
return passthrough_module(f'{parent.__name__}.{attr}',
importlib.import_module(f'.{attr}', child.__name__))
with contextlib.suppress(AttributeError): with contextlib.suppress(AttributeError):
return getattr(child_module, attr) return getattr(child, attr)
if _is_package(child_module):
with contextlib.suppress(ImportError):
return importlib.import_module(f'.{attr}', child)
return _NO_ATTRIBUTE return _NO_ATTRIBUTE
# Python 3.6 does not have module level __getattr__ parent = sys.modules.get(parent, types.ModuleType(parent))
# https://peps.python.org/pep-0562/ parent.__class__ = EnhancedModule
sys.modules[parent].__class__ = PassthroughModule parent.__getattr__ = __getattr__
return parent
+30
View File
@@ -0,0 +1,30 @@
import types
from ..compat import functools
from ..compat.compat_utils import passthrough_module
try:
import Cryptodome as _parent
except ImportError:
try:
import Crypto as _parent
except (ImportError, SyntaxError): # Old Crypto gives SyntaxError in newer Python
_parent = types.ModuleType('no_Cryptodome')
__bool__ = lambda: False
passthrough_module(__name__, _parent, (..., '__version__'))
del passthrough_module
@property
@functools.cache
def _yt_dlp__identifier():
if _parent.__name__ == 'Crypto':
from Crypto.Cipher import AES
try:
# In pycrypto, mode defaults to ECB. See:
# https://www.pycryptodome.org/en/latest/src/vs_pycrypto.html#:~:text=not%20have%20ECB%20as%20default%20mode
AES.new(b'abcdefghijklmnop')
except TypeError:
return 'pycrypto'
return _parent.__name__
@@ -23,24 +23,6 @@ else:
certifi = None certifi = None
try:
from Cryptodome.Cipher import AES as Cryptodome_AES
except ImportError:
try:
from Crypto.Cipher import AES as Cryptodome_AES
except (ImportError, SyntaxError): # Old Crypto gives SyntaxError in newer Python
Cryptodome_AES = None
else:
try:
# In pycrypto, mode defaults to ECB. See:
# https://www.pycryptodome.org/en/latest/src/vs_pycrypto.html#:~:text=not%20have%20ECB%20as%20default%20mode
Cryptodome_AES.new(b'abcdefghijklmnop')
except TypeError:
pass
else:
Cryptodome_AES._yt_dlp__identifier = 'pycrypto'
try: try:
import mutagen import mutagen
except ImportError: except ImportError:
@@ -84,12 +66,16 @@ else:
xattr._yt_dlp__identifier = 'pyxattr' xattr._yt_dlp__identifier = 'pyxattr'
from . import Cryptodome
all_dependencies = {k: v for k, v in globals().items() if not k.startswith('_')} all_dependencies = {k: v for k, v in globals().items() if not k.startswith('_')}
available_dependencies = {k: v for k, v in all_dependencies.items() if v} available_dependencies = {k: v for k, v in all_dependencies.items() if v}
# Deprecated
Cryptodome_AES = Cryptodome.Cipher.AES if Cryptodome else None
__all__ = [ __all__ = [
'all_dependencies', 'all_dependencies',
'available_dependencies', 'available_dependencies',
+5 -7
View File
@@ -15,15 +15,16 @@ from ..minicurses import (
from ..utils import ( from ..utils import (
IDENTITY, IDENTITY,
NO_DEFAULT, NO_DEFAULT,
NUMBER_RE,
LockingUnsupportedError, LockingUnsupportedError,
Namespace, Namespace,
RetryManager, RetryManager,
classproperty, classproperty,
decodeArgument, decodeArgument,
deprecation_warning,
encodeFilename, encodeFilename,
format_bytes, format_bytes,
join_nonempty, join_nonempty,
parse_bytes,
remove_start, remove_start,
sanitize_open, sanitize_open,
shell_quote, shell_quote,
@@ -180,12 +181,9 @@ class FileDownloader:
@staticmethod @staticmethod
def parse_bytes(bytestr): def parse_bytes(bytestr):
"""Parse a string indicating a byte quantity into an integer.""" """Parse a string indicating a byte quantity into an integer."""
matchobj = re.match(rf'(?i)^({NUMBER_RE})([kMGTPEZY]?)$', bytestr) deprecation_warning('yt_dlp.FileDownloader.parse_bytes is deprecated and '
if matchobj is None: 'may be removed in the future. Use yt_dlp.utils.parse_bytes instead')
return None return parse_bytes(bytestr)
number = float(matchobj.group(1))
multiplier = 1024.0 ** 'bkmgtpezy'.index(matchobj.group(2).lower())
return int(round(number * multiplier))
def slow_down(self, start_time, now, byte_counter): def slow_down(self, start_time, now, byte_counter):
"""Sleep if the download speed is over the rate limit.""" """Sleep if the download speed is over the rate limit."""
+11 -3
View File
@@ -1,8 +1,9 @@
import time import time
import urllib.parse
from . import get_suitable_downloader from . import get_suitable_downloader
from .fragment import FragmentFD from .fragment import FragmentFD
from ..utils import urljoin from ..utils import update_url_query, urljoin
class DashSegmentsFD(FragmentFD): class DashSegmentsFD(FragmentFD):
@@ -40,7 +41,12 @@ class DashSegmentsFD(FragmentFD):
self._prepare_and_start_frag_download(ctx, fmt) self._prepare_and_start_frag_download(ctx, fmt)
ctx['start'] = real_start ctx['start'] = real_start
fragments_to_download = self._get_fragments(fmt, ctx) extra_query = None
extra_param_to_segment_url = info_dict.get('extra_param_to_segment_url')
if extra_param_to_segment_url:
extra_query = urllib.parse.parse_qs(extra_param_to_segment_url)
fragments_to_download = self._get_fragments(fmt, ctx, extra_query)
if real_downloader: if real_downloader:
self.to_screen( self.to_screen(
@@ -57,7 +63,7 @@ class DashSegmentsFD(FragmentFD):
fragments = fragments(ctx) if callable(fragments) else fragments fragments = fragments(ctx) if callable(fragments) else fragments
return [next(iter(fragments))] if self.params.get('test') else fragments return [next(iter(fragments))] if self.params.get('test') else fragments
def _get_fragments(self, fmt, ctx): def _get_fragments(self, fmt, ctx, extra_query):
fragment_base_url = fmt.get('fragment_base_url') fragment_base_url = fmt.get('fragment_base_url')
fragments = self._resolve_fragments(fmt['fragments'], ctx) fragments = self._resolve_fragments(fmt['fragments'], ctx)
@@ -70,6 +76,8 @@ class DashSegmentsFD(FragmentFD):
if not fragment_url: if not fragment_url:
assert fragment_base_url assert fragment_base_url
fragment_url = urljoin(fragment_base_url, fragment['path']) fragment_url = urljoin(fragment_base_url, fragment['path'])
if extra_query:
fragment_url = update_url_query(fragment_url, extra_query)
yield { yield {
'frag_index': frag_index, 'frag_index': frag_index,
+120 -22
View File
@@ -1,9 +1,11 @@
import enum import enum
import json
import os.path import os.path
import re import re
import subprocess import subprocess
import sys import sys
import time import time
import uuid
from .fragment import FragmentFD from .fragment import FragmentFD
from ..compat import functools from ..compat import functools
@@ -20,8 +22,10 @@ from ..utils import (
determine_ext, determine_ext,
encodeArgument, encodeArgument,
encodeFilename, encodeFilename,
find_available_port,
handle_youtubedl_headers, handle_youtubedl_headers,
remove_end, remove_end,
sanitized_Request,
traverse_obj, traverse_obj,
) )
@@ -60,7 +64,6 @@ class ExternalFD(FragmentFD):
} }
if filename != '-': if filename != '-':
fsize = os.path.getsize(encodeFilename(tmpfilename)) fsize = os.path.getsize(encodeFilename(tmpfilename))
self.to_screen(f'\r[{self.get_basename()}] Downloaded {fsize} bytes')
self.try_rename(tmpfilename, filename) self.try_rename(tmpfilename, filename)
status.update({ status.update({
'downloaded_bytes': fsize, 'downloaded_bytes': fsize,
@@ -101,6 +104,7 @@ class ExternalFD(FragmentFD):
return all(( return all((
not info_dict.get('to_stdout') or Features.TO_STDOUT in cls.SUPPORTED_FEATURES, not info_dict.get('to_stdout') or Features.TO_STDOUT in cls.SUPPORTED_FEATURES,
'+' not in info_dict['protocol'] or Features.MULTIPLE_FORMATS in cls.SUPPORTED_FEATURES, '+' not in info_dict['protocol'] or Features.MULTIPLE_FORMATS in cls.SUPPORTED_FEATURES,
not traverse_obj(info_dict, ('hls_aes', ...), 'extra_param_to_segment_url'),
all(proto in cls.SUPPORTED_PROTOCOLS for proto in info_dict['protocol'].split('+')), all(proto in cls.SUPPORTED_PROTOCOLS for proto in info_dict['protocol'].split('+')),
)) ))
@@ -129,8 +133,7 @@ class ExternalFD(FragmentFD):
self._debug_cmd(cmd) self._debug_cmd(cmd)
if 'fragments' not in info_dict: if 'fragments' not in info_dict:
_, stderr, returncode = Popen.run( _, stderr, returncode = self._call_process(cmd, info_dict)
cmd, text=True, stderr=subprocess.PIPE if self._CAPTURE_STDERR else None)
if returncode and stderr: if returncode and stderr:
self.to_stderr(stderr) self.to_stderr(stderr)
return returncode return returncode
@@ -140,7 +143,7 @@ class ExternalFD(FragmentFD):
retry_manager = RetryManager(self.params.get('fragment_retries'), self.report_retry, retry_manager = RetryManager(self.params.get('fragment_retries'), self.report_retry,
frag_index=None, fatal=not skip_unavailable_fragments) frag_index=None, fatal=not skip_unavailable_fragments)
for retry in retry_manager: for retry in retry_manager:
_, stderr, returncode = Popen.run(cmd, text=True, stderr=subprocess.PIPE) _, stderr, returncode = self._call_process(cmd, info_dict)
if not returncode: if not returncode:
break break
# TODO: Decide whether to retry based on error code # TODO: Decide whether to retry based on error code
@@ -172,6 +175,9 @@ class ExternalFD(FragmentFD):
self.try_remove(encodeFilename('%s.frag.urls' % tmpfilename)) self.try_remove(encodeFilename('%s.frag.urls' % tmpfilename))
return 0 return 0
def _call_process(self, cmd, info_dict):
return Popen.run(cmd, text=True, stderr=subprocess.PIPE)
class CurlFD(ExternalFD): class CurlFD(ExternalFD):
AVAILABLE_OPT = '-V' AVAILABLE_OPT = '-V'
@@ -256,6 +262,15 @@ class Aria2cFD(ExternalFD):
def _aria2c_filename(fn): def _aria2c_filename(fn):
return fn if os.path.isabs(fn) else f'.{os.path.sep}{fn}' return fn if os.path.isabs(fn) else f'.{os.path.sep}{fn}'
def _call_downloader(self, tmpfilename, info_dict):
# FIXME: Disabled due to https://github.com/yt-dlp/yt-dlp/issues/5931
if False and 'no-external-downloader-progress' not in self.params.get('compat_opts', []):
info_dict['__rpc'] = {
'port': find_available_port() or 19190,
'secret': str(uuid.uuid4()),
}
return super()._call_downloader(tmpfilename, info_dict)
def _make_cmd(self, tmpfilename, info_dict): def _make_cmd(self, tmpfilename, info_dict):
cmd = [self.exe, '-c', cmd = [self.exe, '-c',
'--console-log-level=warn', '--summary-interval=0', '--download-result=hide', '--console-log-level=warn', '--summary-interval=0', '--download-result=hide',
@@ -276,6 +291,12 @@ class Aria2cFD(ExternalFD):
cmd += self._bool_option('--show-console-readout', 'noprogress', 'false', 'true', '=') cmd += self._bool_option('--show-console-readout', 'noprogress', 'false', 'true', '=')
cmd += self._configuration_args() cmd += self._configuration_args()
if '__rpc' in info_dict:
cmd += [
'--enable-rpc',
f'--rpc-listen-port={info_dict["__rpc"]["port"]}',
f'--rpc-secret={info_dict["__rpc"]["secret"]}']
# aria2c strips out spaces from the beginning/end of filenames and paths. # aria2c strips out spaces from the beginning/end of filenames and paths.
# We work around this issue by adding a "./" to the beginning of the # We work around this issue by adding a "./" to the beginning of the
# filename and relative path, and adding a "/" at the end of the path. # filename and relative path, and adding a "/" at the end of the path.
@@ -304,6 +325,88 @@ class Aria2cFD(ExternalFD):
cmd += ['--', info_dict['url']] cmd += ['--', info_dict['url']]
return cmd return cmd
def aria2c_rpc(self, rpc_port, rpc_secret, method, params=()):
# Does not actually need to be UUID, just unique
sanitycheck = str(uuid.uuid4())
d = json.dumps({
'jsonrpc': '2.0',
'id': sanitycheck,
'method': method,
'params': [f'token:{rpc_secret}', *params],
}).encode('utf-8')
request = sanitized_Request(
f'http://localhost:{rpc_port}/jsonrpc',
data=d, headers={
'Content-Type': 'application/json',
'Content-Length': f'{len(d)}',
'Ytdl-request-proxy': '__noproxy__',
})
with self.ydl.urlopen(request) as r:
resp = json.load(r)
assert resp.get('id') == sanitycheck, 'Something went wrong with RPC server'
return resp['result']
def _call_process(self, cmd, info_dict):
if '__rpc' not in info_dict:
return super()._call_process(cmd, info_dict)
send_rpc = functools.partial(self.aria2c_rpc, info_dict['__rpc']['port'], info_dict['__rpc']['secret'])
started = time.time()
fragmented = 'fragments' in info_dict
frag_count = len(info_dict['fragments']) if fragmented else 1
status = {
'filename': info_dict.get('_filename'),
'status': 'downloading',
'elapsed': 0,
'downloaded_bytes': 0,
'fragment_count': frag_count if fragmented else None,
'fragment_index': 0 if fragmented else None,
}
self._hook_progress(status, info_dict)
def get_stat(key, *obj, average=False):
val = tuple(filter(None, map(float, traverse_obj(obj, (..., ..., key))))) or [0]
return sum(val) / (len(val) if average else 1)
with Popen(cmd, text=True, stdout=subprocess.DEVNULL, stderr=subprocess.PIPE) as p:
# Add a small sleep so that RPC client can receive response,
# or the connection stalls infinitely
time.sleep(0.2)
retval = p.poll()
while retval is None:
# We don't use tellStatus as we won't know the GID without reading stdout
# Ref: https://aria2.github.io/manual/en/html/aria2c.html#aria2.tellActive
active = send_rpc('aria2.tellActive')
completed = send_rpc('aria2.tellStopped', [0, frag_count])
downloaded = get_stat('totalLength', completed) + get_stat('completedLength', active)
speed = get_stat('downloadSpeed', active)
total = frag_count * get_stat('totalLength', active, completed, average=True)
if total < downloaded:
total = None
status.update({
'downloaded_bytes': int(downloaded),
'speed': speed,
'total_bytes': None if fragmented else total,
'total_bytes_estimate': total,
'eta': (total - downloaded) / (speed or 1),
'fragment_index': min(frag_count, len(completed) + 1) if fragmented else None,
'elapsed': time.time() - started
})
self._hook_progress(status, info_dict)
if not active and len(completed) >= frag_count:
send_rpc('aria2.shutdown')
retval = p.wait()
break
time.sleep(0.1)
retval = p.poll()
return '', p.stderr.read(), retval
class HttpieFD(ExternalFD): class HttpieFD(ExternalFD):
AVAILABLE_OPT = '--version' AVAILABLE_OPT = '--version'
@@ -342,7 +445,6 @@ class FFmpegFD(ExternalFD):
and cls.can_download(info_dict)) and cls.can_download(info_dict))
def _call_downloader(self, tmpfilename, info_dict): def _call_downloader(self, tmpfilename, info_dict):
urls = [f['url'] for f in info_dict.get('requested_formats', [])] or [info_dict['url']]
ffpp = FFmpegPostProcessor(downloader=self) ffpp = FFmpegPostProcessor(downloader=self)
if not ffpp.available: if not ffpp.available:
self.report_error('m3u8 download detected but ffmpeg could not be found. Please install') self.report_error('m3u8 download detected but ffmpeg could not be found. Please install')
@@ -372,16 +474,6 @@ class FFmpegFD(ExternalFD):
# http://trac.ffmpeg.org/ticket/6125#comment:10 # http://trac.ffmpeg.org/ticket/6125#comment:10
args += ['-seekable', '1' if seekable else '0'] args += ['-seekable', '1' if seekable else '0']
http_headers = None
if info_dict.get('http_headers'):
youtubedl_headers = handle_youtubedl_headers(info_dict['http_headers'])
http_headers = [
# Trailing \r\n after each HTTP header is important to prevent warning from ffmpeg/avconv:
# [http @ 00000000003d2fa0] No trailing CRLF found in HTTP header.
'-headers',
''.join(f'{key}: {val}\r\n' for key, val in youtubedl_headers.items())
]
env = None env = None
proxy = self.params.get('proxy') proxy = self.params.get('proxy')
if proxy: if proxy:
@@ -434,21 +526,26 @@ class FFmpegFD(ExternalFD):
start_time, end_time = info_dict.get('section_start') or 0, info_dict.get('section_end') start_time, end_time = info_dict.get('section_start') or 0, info_dict.get('section_end')
for i, url in enumerate(urls): selected_formats = info_dict.get('requested_formats') or [info_dict]
if http_headers is not None and re.match(r'^https?://', url): for i, fmt in enumerate(selected_formats):
args += http_headers if fmt.get('http_headers') and re.match(r'^https?://', fmt['url']):
headers_dict = handle_youtubedl_headers(fmt['http_headers'])
# Trailing \r\n after each HTTP header is important to prevent warning from ffmpeg/avconv:
# [http @ 00000000003d2fa0] No trailing CRLF found in HTTP header.
args.extend(['-headers', ''.join(f'{key}: {val}\r\n' for key, val in headers_dict.items())])
if start_time: if start_time:
args += ['-ss', str(start_time)] args += ['-ss', str(start_time)]
if end_time: if end_time:
args += ['-t', str(end_time - start_time)] args += ['-t', str(end_time - start_time)]
args += self._configuration_args((f'_i{i + 1}', '_i')) + ['-i', url] args += self._configuration_args((f'_i{i + 1}', '_i')) + ['-i', fmt['url']]
if not (start_time or end_time) or not self.params.get('force_keyframes_at_cuts'): if not (start_time or end_time) or not self.params.get('force_keyframes_at_cuts'):
args += ['-c', 'copy'] args += ['-c', 'copy']
if info_dict.get('requested_formats') or protocol == 'http_dash_segments': if info_dict.get('requested_formats') or protocol == 'http_dash_segments':
for (i, fmt) in enumerate(info_dict.get('requested_formats') or [info_dict]): for i, fmt in enumerate(selected_formats):
stream_number = fmt.get('manifest_stream_number', 0) stream_number = fmt.get('manifest_stream_number', 0)
args.extend(['-map', f'{i}:{stream_number}']) args.extend(['-map', f'{i}:{stream_number}'])
@@ -488,8 +585,9 @@ class FFmpegFD(ExternalFD):
args.append(encodeFilename(ffpp._ffmpeg_filename_argument(tmpfilename), True)) args.append(encodeFilename(ffpp._ffmpeg_filename_argument(tmpfilename), True))
self._debug_cmd(args) self._debug_cmd(args)
piped = any(fmt['url'] in ('-', 'pipe:') for fmt in selected_formats)
with Popen(args, stdin=subprocess.PIPE, env=env) as proc: with Popen(args, stdin=subprocess.PIPE, env=env) as proc:
if url in ('-', 'pipe:'): if piped:
self.on_process_started(proc, proc.stdin) self.on_process_started(proc, proc.stdin)
try: try:
retval = proc.wait() retval = proc.wait()
@@ -499,7 +597,7 @@ class FFmpegFD(ExternalFD):
# produces a file that is playable (this is mostly useful for live # produces a file that is playable (this is mostly useful for live
# streams). Note that Windows is not affected and produces playable # streams). Note that Windows is not affected and produces playable
# files (see https://github.com/ytdl-org/youtube-dl/issues/8300). # files (see https://github.com/ytdl-org/youtube-dl/issues/8300).
if isinstance(e, KeyboardInterrupt) and sys.platform != 'win32' and url not in ('-', 'pipe:'): if isinstance(e, KeyboardInterrupt) and sys.platform != 'win32' and not piped:
proc.communicate_or_kill(b'q') proc.communicate_or_kill(b'q')
else: else:
proc.kill(timeout=None) proc.kill(timeout=None)
+3 -2
View File
@@ -360,7 +360,8 @@ class FragmentFD(FileDownloader):
if not decrypt_info or decrypt_info['METHOD'] != 'AES-128': if not decrypt_info or decrypt_info['METHOD'] != 'AES-128':
return frag_content return frag_content
iv = decrypt_info.get('IV') or struct.pack('>8xq', fragment['media_sequence']) iv = decrypt_info.get('IV') or struct.pack('>8xq', fragment['media_sequence'])
decrypt_info['KEY'] = decrypt_info.get('KEY') or _get_key(info_dict.get('_decryption_key_url') or decrypt_info['URI']) decrypt_info['KEY'] = (decrypt_info.get('KEY')
or _get_key(traverse_obj(info_dict, ('hls_aes', 'uri')) or decrypt_info['URI']))
# Don't decrypt the content in tests since the data is explicitly truncated and it's not to a valid block # Don't decrypt the content in tests since the data is explicitly truncated and it's not to a valid block
# size (see https://github.com/ytdl-org/youtube-dl/pull/27660). Tests only care that the correct data downloaded, # size (see https://github.com/ytdl-org/youtube-dl/pull/27660). Tests only care that the correct data downloaded,
# not what it decrypts to. # not what it decrypts to.
@@ -382,7 +383,7 @@ class FragmentFD(FileDownloader):
max_workers = self.params.get('concurrent_fragment_downloads', 1) max_workers = self.params.get('concurrent_fragment_downloads', 1)
if max_progress > 1: if max_progress > 1:
self._prepare_multiline_status(max_progress) self._prepare_multiline_status(max_progress)
is_live = any(traverse_obj(args, (..., 2, 'is_live'), default=[])) is_live = any(traverse_obj(args, (..., 2, 'is_live')))
def thread_func(idx, ctx, fragments, info_dict, tpe): def thread_func(idx, ctx, fragments, info_dict, tpe):
ctx['max_progress'] = max_progress ctx['max_progress'] = max_progress
+26 -15
View File
@@ -7,8 +7,15 @@ from . import get_suitable_downloader
from .external import FFmpegFD from .external import FFmpegFD
from .fragment import FragmentFD from .fragment import FragmentFD
from .. import webvtt from .. import webvtt
from ..dependencies import Cryptodome_AES from ..dependencies import Cryptodome
from ..utils import bug_reports_message, parse_m3u8_attributes, update_url_query from ..utils import (
bug_reports_message,
parse_m3u8_attributes,
remove_start,
traverse_obj,
update_url_query,
urljoin,
)
class HlsFD(FragmentFD): class HlsFD(FragmentFD):
@@ -63,7 +70,7 @@ class HlsFD(FragmentFD):
can_download, message = self.can_download(s, info_dict, self.params.get('allow_unplayable_formats')), None can_download, message = self.can_download(s, info_dict, self.params.get('allow_unplayable_formats')), None
if can_download: if can_download:
has_ffmpeg = FFmpegFD.available() has_ffmpeg = FFmpegFD.available()
no_crypto = not Cryptodome_AES and '#EXT-X-KEY:METHOD=AES-128' in s no_crypto = not Cryptodome and '#EXT-X-KEY:METHOD=AES-128' in s
if no_crypto and has_ffmpeg: if no_crypto and has_ffmpeg:
can_download, message = False, 'The stream has AES-128 encryption and pycryptodomex is not available' can_download, message = False, 'The stream has AES-128 encryption and pycryptodomex is not available'
elif no_crypto: elif no_crypto:
@@ -150,6 +157,13 @@ class HlsFD(FragmentFD):
i = 0 i = 0
media_sequence = 0 media_sequence = 0
decrypt_info = {'METHOD': 'NONE'} decrypt_info = {'METHOD': 'NONE'}
external_aes_key = traverse_obj(info_dict, ('hls_aes', 'key'))
if external_aes_key:
external_aes_key = binascii.unhexlify(remove_start(external_aes_key, '0x'))
assert len(external_aes_key) in (16, 24, 32), 'Invalid length for HLS AES-128 key'
external_aes_iv = traverse_obj(info_dict, ('hls_aes', 'iv'))
if external_aes_iv:
external_aes_iv = binascii.unhexlify(remove_start(external_aes_iv, '0x').zfill(32))
byte_range = {} byte_range = {}
discontinuity_count = 0 discontinuity_count = 0
frag_index = 0 frag_index = 0
@@ -165,10 +179,7 @@ class HlsFD(FragmentFD):
frag_index += 1 frag_index += 1
if frag_index <= ctx['fragment_index']: if frag_index <= ctx['fragment_index']:
continue continue
frag_url = ( frag_url = urljoin(man_url, line)
line
if re.match(r'^https?://', line)
else urllib.parse.urljoin(man_url, line))
if extra_query: if extra_query:
frag_url = update_url_query(frag_url, extra_query) frag_url = update_url_query(frag_url, extra_query)
@@ -190,10 +201,7 @@ class HlsFD(FragmentFD):
return False return False
frag_index += 1 frag_index += 1
map_info = parse_m3u8_attributes(line[11:]) map_info = parse_m3u8_attributes(line[11:])
frag_url = ( frag_url = urljoin(man_url, map_info.get('URI'))
map_info.get('URI')
if re.match(r'^https?://', map_info.get('URI'))
else urllib.parse.urljoin(man_url, map_info.get('URI')))
if extra_query: if extra_query:
frag_url = update_url_query(frag_url, extra_query) frag_url = update_url_query(frag_url, extra_query)
@@ -218,11 +226,14 @@ class HlsFD(FragmentFD):
decrypt_url = decrypt_info.get('URI') decrypt_url = decrypt_info.get('URI')
decrypt_info = parse_m3u8_attributes(line[11:]) decrypt_info = parse_m3u8_attributes(line[11:])
if decrypt_info['METHOD'] == 'AES-128': if decrypt_info['METHOD'] == 'AES-128':
if 'IV' in decrypt_info: if external_aes_iv:
decrypt_info['IV'] = external_aes_iv
elif 'IV' in decrypt_info:
decrypt_info['IV'] = binascii.unhexlify(decrypt_info['IV'][2:].zfill(32)) decrypt_info['IV'] = binascii.unhexlify(decrypt_info['IV'][2:].zfill(32))
if not re.match(r'^https?://', decrypt_info['URI']): if external_aes_key:
decrypt_info['URI'] = urllib.parse.urljoin( decrypt_info['KEY'] = external_aes_key
man_url, decrypt_info['URI']) else:
decrypt_info['URI'] = urljoin(man_url, decrypt_info['URI'])
if extra_query: if extra_query:
decrypt_info['URI'] = update_url_query(decrypt_info['URI'], extra_query) decrypt_info['URI'] = update_url_query(decrypt_info['URI'], extra_query)
if decrypt_url != decrypt_info['URI']: if decrypt_url != decrypt_info['URI']:
+6 -1
View File
@@ -211,7 +211,12 @@ class HttpFD(FileDownloader):
ctx.stream = None ctx.stream = None
def download(): def download():
data_len = ctx.data.info().get('Content-length', None) data_len = ctx.data.info().get('Content-length')
if ctx.data.info().get('Content-encoding'):
# Content-encoding is present, Content-length is not reliable anymore as we are
# doing auto decompression. (See: https://github.com/yt-dlp/yt-dlp/pull/6176)
data_len = None
# Range HTTP header may be ignored/unsupported by a webserver # Range HTTP header may be ignored/unsupported by a webserver
# (e.g. extractor/scivee.py, extractor/bambuser.py). # (e.g. extractor/scivee.py, extractor/bambuser.py).
+93 -17
View File
@@ -21,7 +21,8 @@ from .youtube import ( # Youtube is moved to the top to improve performance
YoutubeYtBeIE, YoutubeYtBeIE,
YoutubeYtUserIE, YoutubeYtUserIE,
YoutubeWatchLaterIE, YoutubeWatchLaterIE,
YoutubeShortsAudioPivotIE YoutubeShortsAudioPivotIE,
YoutubeConsentRedirectIE,
) )
from .abc import ( from .abc import (
@@ -78,6 +79,8 @@ from .agora import (
WyborczaVideoIE, WyborczaVideoIE,
) )
from .airmozilla import AirMozillaIE from .airmozilla import AirMozillaIE
from .airtv import AirTVIE
from .aitube import AitubeKZVideoIE
from .aljazeera import AlJazeeraIE from .aljazeera import AlJazeeraIE
from .alphaporno import AlphaPornoIE from .alphaporno import AlphaPornoIE
from .amara import AmaraIE from .amara import AmaraIE
@@ -86,11 +89,20 @@ from .alura import (
AluraCourseIE AluraCourseIE
) )
from .amcnetworks import AMCNetworksIE from .amcnetworks import AMCNetworksIE
from .amazon import AmazonStoreIE from .amazon import (
AmazonStoreIE,
AmazonReviewsIE,
)
from .amazonminitv import (
AmazonMiniTVIE,
AmazonMiniTVSeasonIE,
AmazonMiniTVSeriesIE,
)
from .americastestkitchen import ( from .americastestkitchen import (
AmericasTestKitchenIE, AmericasTestKitchenIE,
AmericasTestKitchenSeasonIE, AmericasTestKitchenSeasonIE,
) )
from .anchorfm import AnchorFMEpisodeIE
from .angel import AngelIE from .angel import AngelIE
from .anvato import AnvatoIE from .anvato import AnvatoIE
from .aol import AolIE from .aol import AolIE
@@ -111,6 +123,7 @@ from .applepodcasts import ApplePodcastsIE
from .archiveorg import ( from .archiveorg import (
ArchiveOrgIE, ArchiveOrgIE,
YoutubeWebArchiveIE, YoutubeWebArchiveIE,
VLiveWebArchiveIE,
) )
from .arcpublishing import ArcPublishingIE from .arcpublishing import ArcPublishingIE
from .arkena import ArkenaIE from .arkena import ArkenaIE
@@ -178,6 +191,10 @@ from .bbc import (
from .beeg import BeegIE from .beeg import BeegIE
from .behindkink import BehindKinkIE from .behindkink import BehindKinkIE
from .bellmedia import BellMediaIE from .bellmedia import BellMediaIE
from .beatbump import (
BeatBumpVideoIE,
BeatBumpPlaylistIE,
)
from .beatport import BeatportIE from .beatport import BeatportIE
from .berufetv import BerufeTVIE from .berufetv import BerufeTVIE
from .bet import BetIE from .bet import BetIE
@@ -222,12 +239,14 @@ from .bleacherreport import (
BleacherReportIE, BleacherReportIE,
BleacherReportCMSIE, BleacherReportCMSIE,
) )
from .blerp import BlerpIE
from .blogger import BloggerIE from .blogger import BloggerIE
from .bloomberg import BloombergIE from .bloomberg import BloombergIE
from .bokecc import BokeCCIE from .bokecc import BokeCCIE
from .bongacams import BongaCamsIE from .bongacams import BongaCamsIE
from .bostonglobe import BostonGlobeIE from .bostonglobe import BostonGlobeIE
from .box import BoxIE from .box import BoxIE
from .boxcast import BoxCastVideoIE
from .booyah import BooyahClipsIE from .booyah import BooyahClipsIE
from .bpb import BpbIE from .bpb import BpbIE
from .br import ( from .br import (
@@ -461,6 +480,8 @@ from .drtuber import DrTuberIE
from .drtv import ( from .drtv import (
DRTVIE, DRTVIE,
DRTVLiveIE, DRTVLiveIE,
DRTVSeasonIE,
DRTVSeriesIE,
) )
from .dtube import DTubeIE from .dtube import DTubeIE
from .dvtv import DVTVIE from .dvtv import DVTVIE
@@ -489,6 +510,7 @@ from .dw import (
) )
from .eagleplatform import EaglePlatformIE, ClipYouEmbedIE from .eagleplatform import EaglePlatformIE, ClipYouEmbedIE
from .ebaumsworld import EbaumsWorldIE from .ebaumsworld import EbaumsWorldIE
from .ebay import EbayIE
from .echomsk import EchoMskIE from .echomsk import EchoMskIE
from .egghead import ( from .egghead import (
EggheadCourseIE, EggheadCourseIE,
@@ -531,7 +553,7 @@ from .espn import (
ESPNCricInfoIE, ESPNCricInfoIE,
) )
from .esri import EsriVideoIE from .esri import EsriVideoIE
from .europa import EuropaIE from .europa import EuropaIE, EuroParlWebstreamIE
from .europeantour import EuropeanTourIE from .europeantour import EuropeanTourIE
from .eurosport import EurosportIE from .eurosport import EurosportIE
from .euscreen import EUScreenIE from .euscreen import EUScreenIE
@@ -727,6 +749,7 @@ from .hungama import (
HungamaAlbumPlaylistIE, HungamaAlbumPlaylistIE,
) )
from .hypem import HypemIE from .hypem import HypemIE
from .hypergryph import MonsterSirenHypergryphMusicIE
from .hytale import HytaleIE from .hytale import HytaleIE
from .icareus import IcareusIE from .icareus import IcareusIE
from .ichinanalive import ( from .ichinanalive import (
@@ -820,6 +843,8 @@ from .joj import JojIE
from .jwplatform import JWPlatformIE from .jwplatform import JWPlatformIE
from .kakao import KakaoIE from .kakao import KakaoIE
from .kaltura import KalturaIE from .kaltura import KalturaIE
from .kanal2 import Kanal2IE
from .kankanews import KankaNewsIE
from .karaoketv import KaraoketvIE from .karaoketv import KaraoketvIE
from .karrierevideos import KarriereVideosIE from .karrierevideos import KarriereVideosIE
from .keezmovies import KeezMoviesIE from .keezmovies import KeezMoviesIE
@@ -829,10 +854,15 @@ from .khanacademy import (
KhanAcademyIE, KhanAcademyIE,
KhanAcademyUnitIE, KhanAcademyUnitIE,
) )
from .kick import (
KickIE,
KickVODIE,
)
from .kicker import KickerIE from .kicker import KickerIE
from .kickstarter import KickStarterIE from .kickstarter import KickStarterIE
from .kinja import KinjaEmbedIE from .kinja import KinjaEmbedIE
from .kinopoisk import KinoPoiskIE from .kinopoisk import KinoPoiskIE
from .kommunetv import KommunetvIE
from .kompas import KompasVideoIE from .kompas import KompasVideoIE
from .konserthusetplay import KonserthusetPlayIE from .konserthusetplay import KonserthusetPlayIE
from .koo import KooIE from .koo import KooIE
@@ -976,6 +1006,10 @@ from .mediasite import (
MediasiteCatalogIE, MediasiteCatalogIE,
MediasiteNamedCatalogIE, MediasiteNamedCatalogIE,
) )
from .mediastream import (
MediaStreamIE,
WinSportsVideoIE,
)
from .mediaworksnz import MediaWorksNZVODIE from .mediaworksnz import MediaWorksNZVODIE
from .medici import MediciIE from .medici import MediciIE
from .megaphone import MegaphoneIE from .megaphone import MegaphoneIE
@@ -1144,6 +1178,7 @@ from .neteasemusic import (
from .netverse import ( from .netverse import (
NetverseIE, NetverseIE,
NetversePlaylistIE, NetversePlaylistIE,
NetverseSearchIE,
) )
from .newgrounds import ( from .newgrounds import (
NewgroundsIE, NewgroundsIE,
@@ -1168,6 +1203,8 @@ from .nfhsnetwork import NFHSNetworkIE
from .nfl import ( from .nfl import (
NFLIE, NFLIE,
NFLArticleIE, NFLArticleIE,
NFLPlusEpisodeIE,
NFLPlusReplayIE,
) )
from .nhk import ( from .nhk import (
NhkVodIE, NhkVodIE,
@@ -1205,6 +1242,7 @@ from .nintendo import NintendoIE
from .nitter import NitterIE from .nitter import NitterIE
from .njpwworld import NJPWWorldIE from .njpwworld import NJPWWorldIE
from .nobelprize import NobelPrizeIE from .nobelprize import NobelPrizeIE
from .noice import NoicePodcastIE
from .nonktube import NonkTubeIE from .nonktube import NonkTubeIE
from .noodlemagazine import NoodleMagazineIE from .noodlemagazine import NoodleMagazineIE
from .noovo import NoovoIE from .noovo import NoovoIE
@@ -1257,8 +1295,10 @@ from .nytimes import (
) )
from .nuvid import NuvidIE from .nuvid import NuvidIE
from .nzherald import NZHeraldIE from .nzherald import NZHeraldIE
from .nzonscreen import NZOnScreenIE
from .nzz import NZZIE from .nzz import NZZIE
from .odatv import OdaTVIE from .odatv import OdaTVIE
from .odkmedia import OnDemandChinaEpisodeIE
from .odnoklassniki import OdnoklassnikiIE from .odnoklassniki import OdnoklassnikiIE
from .oftv import ( from .oftv import (
OfTVIE, OfTVIE,
@@ -1270,6 +1310,7 @@ from .on24 import On24IE
from .ondemandkorea import OnDemandKoreaIE from .ondemandkorea import OnDemandKoreaIE
from .onefootball import OneFootballIE from .onefootball import OneFootballIE
from .onenewsnz import OneNewsNZIE from .onenewsnz import OneNewsNZIE
from .oneplace import OnePlacePodcastIE
from .onet import ( from .onet import (
OnetIE, OnetIE,
OnetChannelIE, OnetChannelIE,
@@ -1392,6 +1433,8 @@ from .pokergo import (
from .polsatgo import PolsatGoIE from .polsatgo import PolsatGoIE
from .polskieradio import ( from .polskieradio import (
PolskieRadioIE, PolskieRadioIE,
PolskieRadioLegacyIE,
PolskieRadioAuditionIE,
PolskieRadioCategoryIE, PolskieRadioCategoryIE,
PolskieRadioPlayerIE, PolskieRadioPlayerIE,
PolskieRadioPodcastIE, PolskieRadioPodcastIE,
@@ -1419,6 +1462,7 @@ from .puhutv import (
PuhuTVIE, PuhuTVIE,
PuhuTVSerieIE, PuhuTVSerieIE,
) )
from .pr0gramm import Pr0grammStaticIE, Pr0grammIE
from .prankcast import PrankCastIE from .prankcast import PrankCastIE
from .premiershiprugby import PremiershipRugbyIE from .premiershiprugby import PremiershipRugbyIE
from .presstv import PressTVIE from .presstv import PressTVIE
@@ -1480,6 +1524,10 @@ from .raywenderlich import (
RayWenderlichCourseIE, RayWenderlichCourseIE,
) )
from .rbmaradio import RBMARadioIE from .rbmaradio import RBMARadioIE
from .rbgtum import (
RbgTumIE,
RbgTumCourseIE,
)
from .rcs import ( from .rcs import (
RCSIE, RCSIE,
RCSEmbedsIE, RCSEmbedsIE,
@@ -1524,7 +1572,10 @@ from .rokfin import (
) )
from .roosterteeth import RoosterTeethIE, RoosterTeethSeriesIE from .roosterteeth import RoosterTeethIE, RoosterTeethSeriesIE
from .rottentomatoes import RottenTomatoesIE from .rottentomatoes import RottenTomatoesIE
from .rozhlas import RozhlasIE from .rozhlas import (
RozhlasIE,
RozhlasVltavaIE,
)
from .rte import RteIE, RteRadioIE from .rte import RteIE, RteRadioIE
from .rtlnl import ( from .rtlnl import (
RtlNlIE, RtlNlIE,
@@ -1561,6 +1612,7 @@ from .ruhd import RUHDIE
from .rule34video import Rule34VideoIE from .rule34video import Rule34VideoIE
from .rumble import ( from .rumble import (
RumbleEmbedIE, RumbleEmbedIE,
RumbleIE,
RumbleChannelIE, RumbleChannelIE,
) )
from .rutube import ( from .rutube import (
@@ -1603,6 +1655,7 @@ from .savefrom import SaveFromIE
from .sbs import SBSIE from .sbs import SBSIE
from .screen9 import Screen9IE from .screen9 import Screen9IE
from .screencast import ScreencastIE from .screencast import ScreencastIE
from .screencastify import ScreencastifyIE
from .screencastomatic import ScreencastOMaticIE from .screencastomatic import ScreencastOMaticIE
from .scrippsnetworks import ( from .scrippsnetworks import (
ScrippsNetworksWatchIE, ScrippsNetworksWatchIE,
@@ -1632,6 +1685,7 @@ from .shared import (
VivoIE, VivoIE,
) )
from .sharevideos import ShareVideosEmbedIE from .sharevideos import ShareVideosEmbedIE
from .sibnet import SibnetEmbedIE
from .shemaroome import ShemarooMeIE from .shemaroome import ShemarooMeIE
from .showroomlive import ShowRoomLiveIE from .showroomlive import ShowRoomLiveIE
from .simplecast import ( from .simplecast import (
@@ -1679,6 +1733,7 @@ from .soundcloud import (
SoundcloudSetIE, SoundcloudSetIE,
SoundcloudRelatedIE, SoundcloudRelatedIE,
SoundcloudUserIE, SoundcloudUserIE,
SoundcloudUserPermalinkIE,
SoundcloudTrackStationIE, SoundcloudTrackStationIE,
SoundcloudPlaylistIE, SoundcloudPlaylistIE,
SoundcloudSearchIE, SoundcloudSearchIE,
@@ -1810,7 +1865,7 @@ from .telequebec import (
) )
from .teletask import TeleTaskIE from .teletask import TeleTaskIE
from .telewebion import TelewebionIE from .telewebion import TelewebionIE
from .tempo import TempoIE from .tempo import TempoIE, IVXPlayerIE
from .tencent import ( from .tencent import (
IflixEpisodeIE, IflixEpisodeIE,
IflixSeriesIE, IflixSeriesIE,
@@ -1840,6 +1895,11 @@ from .theweatherchannel import TheWeatherChannelIE
from .thisamericanlife import ThisAmericanLifeIE from .thisamericanlife import ThisAmericanLifeIE
from .thisav import ThisAVIE from .thisav import ThisAVIE
from .thisoldhouse import ThisOldHouseIE from .thisoldhouse import ThisOldHouseIE
from .thisvid import (
ThisVidIE,
ThisVidMemberIE,
ThisVidPlaylistIE,
)
from .threespeak import ( from .threespeak import (
ThreeSpeakIE, ThreeSpeakIE,
ThreeSpeakUserIE, ThreeSpeakUserIE,
@@ -1852,6 +1912,7 @@ from .tiktok import (
TikTokEffectIE, TikTokEffectIE,
TikTokTagIE, TikTokTagIE,
TikTokVMIE, TikTokVMIE,
TikTokLiveIE,
DouyinIE, DouyinIE,
) )
from .tinypic import TinyPicIE from .tinypic import TinyPicIE
@@ -1889,6 +1950,7 @@ from .trovo import (
TrovoChannelVodIE, TrovoChannelVodIE,
TrovoChannelClipIE, TrovoChannelClipIE,
) )
from .trtcocuk import TrtCocukVideoIE
from .trueid import TrueIDIE from .trueid import TrueIDIE
from .trunews import TruNewsIE from .trunews import TruNewsIE
from .truth import TruthIE from .truth import TruthIE
@@ -2002,6 +2064,10 @@ from .twitter import (
TwitterSpacesIE, TwitterSpacesIE,
TwitterShortenerIE, TwitterShortenerIE,
) )
from .txxx import (
TxxxIE,
PornTopIE,
)
from .udemy import ( from .udemy import (
UdemyIE, UdemyIE,
UdemyCourseIE UdemyCourseIE
@@ -2043,7 +2109,10 @@ from .varzesh3 import Varzesh3IE
from .vbox7 import Vbox7IE from .vbox7 import Vbox7IE
from .veehd import VeeHDIE from .veehd import VeeHDIE
from .veo import VeoIE from .veo import VeoIE
from .veoh import VeohIE from .veoh import (
VeohIE,
VeohUserIE
)
from .vesti import VestiIE from .vesti import VestiIE
from .vevo import ( from .vevo import (
VevoIE, VevoIE,
@@ -2069,6 +2138,13 @@ from .videocampus_sachsen import (
) )
from .videodetective import VideoDetectiveIE from .videodetective import VideoDetectiveIE
from .videofyme import VideofyMeIE from .videofyme import VideofyMeIE
from .videoken import (
VideoKenIE,
VideoKenPlayerIE,
VideoKenPlaylistIE,
VideoKenCategoryIE,
VideoKenTopicIE,
)
from .videomore import ( from .videomore import (
VideomoreIE, VideomoreIE,
VideomoreVideoIE, VideomoreVideoIE,
@@ -2093,6 +2169,7 @@ from .vimeo import (
VimeoGroupsIE, VimeoGroupsIE,
VimeoLikesIE, VimeoLikesIE,
VimeoOndemandIE, VimeoOndemandIE,
VimeoProIE,
VimeoReviewIE, VimeoReviewIE,
VimeoUserIE, VimeoUserIE,
VimeoWatchLaterIE, VimeoWatchLaterIE,
@@ -2116,17 +2193,14 @@ from .viu import (
ViuIE, ViuIE,
ViuPlaylistIE, ViuPlaylistIE,
ViuOTTIE, ViuOTTIE,
ViuOTTIndonesiaIE,
) )
from .vk import ( from .vk import (
VKIE, VKIE,
VKUserVideosIE, VKUserVideosIE,
VKWallPostIE, VKWallPostIE,
) )
from .vlive import ( from .vocaroo import VocarooIE
VLiveIE,
VLivePostIE,
VLiveChannelIE,
)
from .vodlocker import VodlockerIE from .vodlocker import VodlockerIE
from .vodpl import VODPlIE from .vodpl import VODPlIE
from .vodplatform import VODPlatformIE from .vodplatform import VODPlatformIE
@@ -2135,6 +2209,7 @@ from .voicy import (
VoicyIE, VoicyIE,
VoicyChannelIE, VoicyChannelIE,
) )
from .volejtv import VolejTVIE
from .voot import ( from .voot import (
VootIE, VootIE,
VootSeriesIE, VootSeriesIE,
@@ -2180,6 +2255,7 @@ from .wdr import (
WDRElefantIE, WDRElefantIE,
WDRMobileIE, WDRMobileIE,
) )
from .webcamerapl import WebcameraplIE
from .webcaster import ( from .webcaster import (
WebcasterIE, WebcasterIE,
WebcasterFeedIE, WebcasterFeedIE,
@@ -2211,11 +2287,16 @@ from .wppilot import (
WPPilotIE, WPPilotIE,
WPPilotChannelsIE, WPPilotChannelsIE,
) )
from .wrestleuniverse import (
WrestleUniverseVODIE,
WrestleUniversePPVIE,
)
from .wsj import ( from .wsj import (
WSJIE, WSJIE,
WSJArticleIE, WSJArticleIE,
) )
from .wwe import WWEIE from .wwe import WWEIE
from .xanimu import XanimuIE
from .xbef import XBefIE from .xbef import XBefIE
from .xboxclips import XboxClipsIE from .xboxclips import XboxClipsIE
from .xfileshare import XFileShareIE from .xfileshare import XFileShareIE
@@ -2224,12 +2305,6 @@ from .xhamster import (
XHamsterEmbedIE, XHamsterEmbedIE,
XHamsterUserIE, XHamsterUserIE,
) )
from .xiami import (
XiamiSongIE,
XiamiAlbumIE,
XiamiArtistIE,
XiamiCollectionIE
)
from .ximalaya import ( from .ximalaya import (
XimalayaIE, XimalayaIE,
XimalayaAlbumIE XimalayaAlbumIE
@@ -2264,6 +2339,7 @@ from .yandexvideo import (
ZenYandexChannelIE, ZenYandexChannelIE,
) )
from .yapfiles import YapFilesIE from .yapfiles import YapFilesIE
from .yappy import YappyIE
from .yesjapan import YesJapanIE from .yesjapan import YesJapanIE
from .yinyuetai import YinYueTaiIE from .yinyuetai import YinYueTaiIE
from .yle_areena import YleAreenaIE from .yle_areena import YleAreenaIE
-3
View File
@@ -155,8 +155,6 @@ class ABCIE(InfoExtractor):
'format_id': format_id 'format_id': format_id
}) })
self._sort_formats(formats)
return { return {
'id': video_id, 'id': video_id,
'title': self._og_search_title(webpage), 'title': self._og_search_title(webpage),
@@ -221,7 +219,6 @@ class ABCIViewIE(InfoExtractor):
entry_protocol='m3u8_native', m3u8_id='hls', fatal=False) entry_protocol='m3u8_native', m3u8_id='hls', fatal=False)
if formats: if formats:
break break
self._sort_formats(formats)
subtitles = {} subtitles = {}
src_vtt = stream.get('captions', {}).get('src-vtt') src_vtt = stream.get('captions', {}).get('src-vtt')
-2
View File
@@ -78,7 +78,6 @@ class ABCOTVSIE(InfoExtractor):
'url': mp4_url, 'url': mp4_url,
'width': 640, 'width': 640,
}) })
self._sort_formats(formats)
image = video.get('image') or {} image = video.get('image') or {}
@@ -119,7 +118,6 @@ class ABCOTVSClipsIE(InfoExtractor):
title = video_data['title'] title = video_data['title']
formats = self._extract_m3u8_formats( formats = self._extract_m3u8_formats(
video_data['videoURL'].split('?')[0], video_id, 'mp4') video_data['videoURL'].split('?')[0], video_id, 'mp4')
self._sort_formats(formats)
return { return {
'id': video_id, 'id': video_id,
+19 -3
View File
@@ -156,7 +156,7 @@ class AbemaTVBaseIE(InfoExtractor):
def _generate_aks(cls, deviceid): def _generate_aks(cls, deviceid):
deviceid = deviceid.encode('utf-8') deviceid = deviceid.encode('utf-8')
# add 1 hour and then drop minute and secs # add 1 hour and then drop minute and secs
ts_1hour = int((time_seconds(hours=9) // 3600 + 1) * 3600) ts_1hour = int((time_seconds() // 3600 + 1) * 3600)
time_struct = time.gmtime(ts_1hour) time_struct = time.gmtime(ts_1hour)
ts_1hour_str = str(ts_1hour).encode('utf-8') ts_1hour_str = str(ts_1hour).encode('utf-8')
@@ -190,6 +190,16 @@ class AbemaTVBaseIE(InfoExtractor):
if self._USERTOKEN: if self._USERTOKEN:
return self._USERTOKEN return self._USERTOKEN
username, _ = self._get_login_info()
AbemaTVBaseIE._USERTOKEN = username and self.cache.load(self._NETRC_MACHINE, username)
if AbemaTVBaseIE._USERTOKEN:
# try authentication with locally stored token
try:
self._get_media_token(True)
return
except ExtractorError as e:
self.report_warning(f'Failed to login with cached user token; obtaining a fresh one ({e})')
AbemaTVBaseIE._DEVICE_ID = str(uuid.uuid4()) AbemaTVBaseIE._DEVICE_ID = str(uuid.uuid4())
aks = self._generate_aks(self._DEVICE_ID) aks = self._generate_aks(self._DEVICE_ID)
user_data = self._download_json( user_data = self._download_json(
@@ -300,6 +310,11 @@ class AbemaTVIE(AbemaTVBaseIE):
_TIMETABLE = None _TIMETABLE = None
def _perform_login(self, username, password): def _perform_login(self, username, password):
self._get_device_token()
if self.cache.load(self._NETRC_MACHINE, username) and self._get_media_token():
self.write_debug('Skipping logging in')
return
if '@' in username: # don't strictly check if it's email address or not if '@' in username: # don't strictly check if it's email address or not
ep, method = 'user/email', 'email' ep, method = 'user/email', 'email'
else: else:
@@ -319,6 +334,7 @@ class AbemaTVIE(AbemaTVBaseIE):
AbemaTVBaseIE._USERTOKEN = login_response['token'] AbemaTVBaseIE._USERTOKEN = login_response['token']
self._get_media_token(True) self._get_media_token(True)
self.cache.store(self._NETRC_MACHINE, username, AbemaTVBaseIE._USERTOKEN)
def _real_extract(self, url): def _real_extract(self, url):
# starting download using infojson from this extractor is undefined behavior, # starting download using infojson from this extractor is undefined behavior,
@@ -416,7 +432,7 @@ class AbemaTVIE(AbemaTVBaseIE):
f'https://api.abema.io/v1/video/programs/{video_id}', video_id, f'https://api.abema.io/v1/video/programs/{video_id}', video_id,
note='Checking playability', note='Checking playability',
headers=headers) headers=headers)
ondemand_types = traverse_obj(api_response, ('terms', ..., 'onDemandType'), default=[]) ondemand_types = traverse_obj(api_response, ('terms', ..., 'onDemandType'))
if 3 not in ondemand_types: if 3 not in ondemand_types:
# cannot acquire decryption key for these streams # cannot acquire decryption key for these streams
self.report_warning('This is a premium-only stream') self.report_warning('This is a premium-only stream')
@@ -489,7 +505,7 @@ class AbemaTVTitleIE(AbemaTVBaseIE):
}) })
yield from ( yield from (
self.url_result(f'https://abema.tv/video/episode/{x}') self.url_result(f'https://abema.tv/video/episode/{x}')
for x in traverse_obj(programs, ('programs', ..., 'id'), default=[])) for x in traverse_obj(programs, ('programs', ..., 'id')))
def _entries(self, playlist_id, series_version): def _entries(self, playlist_id, series_version):
return OnDemandPagedList( return OnDemandPagedList(
-1
View File
@@ -27,7 +27,6 @@ class AcFunVideoBaseIE(InfoExtractor):
**parse_codecs(video.get('codecs', '')) **parse_codecs(video.get('codecs', ''))
}) })
self._sort_formats(formats)
return { return {
'id': video_id, 'id': video_id,
'formats': formats, 'formats': formats,
+1 -2
View File
@@ -168,7 +168,7 @@ Format: Marked,Start,End,Style,Name,MarginL,MarginR,MarginV,Effect,Text'''
}, data=b'')['token'] }, data=b'')['token']
links_url = try_get(options, lambda x: x['video']['url']) or (video_base_url + 'link') links_url = try_get(options, lambda x: x['video']['url']) or (video_base_url + 'link')
self._K = ''.join([random.choice('0123456789abcdef') for _ in range(16)]) self._K = ''.join(random.choices('0123456789abcdef', k=16))
message = bytes_to_intlist(json.dumps({ message = bytes_to_intlist(json.dumps({
'k': self._K, 'k': self._K,
't': token, 't': token,
@@ -235,7 +235,6 @@ Format: Marked,Start,End,Style,Name,MarginL,MarginR,MarginV,Effect,Text'''
for f in m3u8_formats: for f in m3u8_formats:
f['language'] = 'fr' f['language'] = 'fr'
formats.extend(m3u8_formats) formats.extend(m3u8_formats)
self._sort_formats(formats)
video = (self._download_json( video = (self._download_json(
self._API_BASE_URL + 'video/%s' % video_id, video_id, self._API_BASE_URL + 'video/%s' % video_id, video_id,
+1 -1
View File
@@ -1352,7 +1352,7 @@ MSO_INFO = {
} }
class AdobePassIE(InfoExtractor): class AdobePassIE(InfoExtractor): # XXX: Conventionally, base classes should end with BaseIE/InfoExtractor
_SERVICE_PROVIDER_TEMPLATE = 'https://sp.auth.adobe.com/adobe-services/%s' _SERVICE_PROVIDER_TEMPLATE = 'https://sp.auth.adobe.com/adobe-services/%s'
_USER_AGENT = 'Mozilla/5.0 (X11; Linux i686; rv:47.0) Gecko/20100101 Firefox/47.0' _USER_AGENT = 'Mozilla/5.0 (X11; Linux i686; rv:47.0) Gecko/20100101 Firefox/47.0'
_MVPD_CACHE = 'ap-mvpd' _MVPD_CACHE = 'ap-mvpd'
-2
View File
@@ -70,7 +70,6 @@ class AdobeTVBaseIE(InfoExtractor):
}) })
s3_extracted = True s3_extracted = True
formats.append(f) formats.append(f)
self._sort_formats(formats)
return { return {
'id': video_id, 'id': video_id,
@@ -269,7 +268,6 @@ class AdobeTVVideoIE(AdobeTVBaseIE):
'width': int_or_none(source.get('width') or None), 'width': int_or_none(source.get('width') or None),
'url': source_src, 'url': source_src,
}) })
self._sort_formats(formats)
# For both metadata and downloaded files the duration varies among # For both metadata and downloaded files the duration varies among
# formats. I just pick the max one # formats. I just pick the max one
-1
View File
@@ -180,7 +180,6 @@ class AdultSwimIE(TurnerBaseIE):
info['subtitles'].setdefault('en', []).append({ info['subtitles'].setdefault('en', []).append({
'url': asset_url, 'url': asset_url,
}) })
self._sort_formats(info['formats'])
return info return info
else: else:
+1 -3
View File
@@ -8,7 +8,7 @@ from ..utils import (
) )
class AENetworksBaseIE(ThePlatformIE): class AENetworksBaseIE(ThePlatformIE): # XXX: Do not subclass from concrete IE
_BASE_URL_REGEX = r'''(?x)https?:// _BASE_URL_REGEX = r'''(?x)https?://
(?:(?:www|play|watch)\.)? (?:(?:www|play|watch)\.)?
(?P<domain> (?P<domain>
@@ -62,7 +62,6 @@ class AENetworksBaseIE(ThePlatformIE):
subtitles = self._merge_subtitles(subtitles, tp_subtitles) subtitles = self._merge_subtitles(subtitles, tp_subtitles)
if last_e and not formats: if last_e and not formats:
raise last_e raise last_e
self._sort_formats(formats)
return { return {
'id': video_id, 'id': video_id,
'formats': formats, 'formats': formats,
@@ -304,7 +303,6 @@ class HistoryTopicIE(AENetworksBaseIE):
class HistoryPlayerIE(AENetworksBaseIE): class HistoryPlayerIE(AENetworksBaseIE):
IE_NAME = 'history:player' IE_NAME = 'history:player'
_VALID_URL = r'https?://(?:www\.)?(?P<domain>(?:history|biography)\.com)/player/(?P<id>\d+)' _VALID_URL = r'https?://(?:www\.)?(?P<domain>(?:history|biography)\.com)/player/(?P<id>\d+)'
_TESTS = []
def _real_extract(self, url): def _real_extract(self, url):
domain, video_id = self._match_valid_url(url).groups() domain, video_id = self._match_valid_url(url).groups()
+1 -4
View File
@@ -338,7 +338,6 @@ class AfreecaTVIE(InfoExtractor):
}] }]
if not formats and not self.get_param('ignore_no_formats'): if not formats and not self.get_param('ignore_no_formats'):
continue continue
self._sort_formats(formats)
file_info = common_entry.copy() file_info = common_entry.copy()
file_info.update({ file_info.update({
'id': format_id, 'id': format_id,
@@ -380,7 +379,7 @@ class AfreecaTVIE(InfoExtractor):
return info return info
class AfreecaTVLiveIE(AfreecaTVIE): class AfreecaTVLiveIE(AfreecaTVIE): # XXX: Do not subclass from concrete IE
IE_NAME = 'afreecatv:live' IE_NAME = 'afreecatv:live'
_VALID_URL = r'https?://play\.afreeca(?:tv)?\.com/(?P<id>[^/]+)(?:/(?P<bno>\d+))?' _VALID_URL = r'https?://play\.afreeca(?:tv)?\.com/(?P<id>[^/]+)(?:/(?P<bno>\d+))?'
@@ -464,8 +463,6 @@ class AfreecaTVLiveIE(AfreecaTVIE):
'quality': quality_key(quality_str), 'quality': quality_key(quality_str),
}) })
self._sort_formats(formats)
station_info = self._download_json( station_info = self._download_json(
'https://st.afreecatv.com/api/get_station_status.php', broadcast_no, 'https://st.afreecatv.com/api/get_station_status.php', broadcast_no,
query={'szBjId': broadcaster_id}, fatal=False, query={'szBjId': broadcaster_id}, fatal=False,
-2
View File
@@ -55,7 +55,6 @@ class WyborczaVideoIE(InfoExtractor):
if meta['files'].get('dash'): if meta['files'].get('dash'):
formats.extend(self._extract_mpd_formats(base_url + meta['files']['dash'], video_id)) formats.extend(self._extract_mpd_formats(base_url + meta['files']['dash'], video_id))
self._sort_formats(formats)
return { return {
'id': video_id, 'id': video_id,
'formats': formats, 'formats': formats,
@@ -179,7 +178,6 @@ class TokFMPodcastIE(InfoExtractor):
'acodec': ext, 'acodec': ext,
}) })
self._sort_formats(formats)
return { return {
'id': media_id, 'id': media_id,
'formats': formats, 'formats': formats,
+96
View File
@@ -0,0 +1,96 @@
from .common import InfoExtractor
from .youtube import YoutubeIE
from ..utils import (
determine_ext,
int_or_none,
mimetype2ext,
parse_iso8601,
traverse_obj
)
class AirTVIE(InfoExtractor):
_VALID_URL = r'https?://www\.air\.tv/watch\?v=(?P<id>\w+)'
_TESTS = [{
# without youtube_id
'url': 'https://www.air.tv/watch?v=W87jcWleSn2hXZN47zJZsQ',
'info_dict': {
'id': 'W87jcWleSn2hXZN47zJZsQ',
'ext': 'mp4',
'release_date': '20221003',
'release_timestamp': 1664792603,
'channel_id': 'vgfManQlRQKgoFQ8i8peFQ',
'title': 'md5:c12d49ed367c3dadaa67659aff43494c',
'upload_date': '20221003',
'duration': 151,
'view_count': int,
'thumbnail': 'https://cdn-sp-gcs.air.tv/videos/W/8/W87jcWleSn2hXZN47zJZsQ/b13fc56464f47d9d62a36d110b9b5a72-4096x2160_9.jpg',
'timestamp': 1664792603,
}
}, {
# with youtube_id
'url': 'https://www.air.tv/watch?v=sv57EC8tRXG6h8dNXFUU1Q',
'info_dict': {
'id': '2ZTqmpee-bQ',
'ext': 'mp4',
'comment_count': int,
'tags': 'count:11',
'channel_follower_count': int,
'like_count': int,
'uploader': 'Newsflare',
'thumbnail': 'https://i.ytimg.com/vi_webp/2ZTqmpee-bQ/maxresdefault.webp',
'availability': 'public',
'title': 'Geese Chase Alligator Across Golf Course',
'uploader_id': 'NewsflareBreaking',
'channel_url': 'https://www.youtube.com/channel/UCzSSoloGEz10HALUAbYhngQ',
'description': 'md5:99b21d9cea59330149efbd9706e208f5',
'age_limit': 0,
'channel_id': 'UCzSSoloGEz10HALUAbYhngQ',
'uploader_url': 'http://www.youtube.com/user/NewsflareBreaking',
'view_count': int,
'categories': ['News & Politics'],
'live_status': 'not_live',
'playable_in_embed': True,
'channel': 'Newsflare',
'duration': 37,
'upload_date': '20180511',
}
}]
def _get_formats_and_subtitle(self, json_data, video_id):
formats, subtitles = [], {}
for source in traverse_obj(json_data, 'sources', 'sources_desktop', ...):
ext = determine_ext(source.get('src'), mimetype2ext(source.get('type')))
if ext == 'm3u8':
fmts, subs = self._extract_m3u8_formats_and_subtitles(source.get('src'), video_id)
formats.extend(fmts)
self._merge_subtitles(subs, target=subtitles)
else:
formats.append({'url': source.get('src'), 'ext': ext})
return formats, subtitles
def _real_extract(self, url):
display_id = self._match_id(url)
webpage = self._download_webpage(url, display_id)
nextjs_json = self._search_nextjs_data(webpage, display_id)['props']['pageProps']['initialState']['videos'][display_id]
if nextjs_json.get('youtube_id'):
return self.url_result(
f'https://www.youtube.com/watch?v={nextjs_json.get("youtube_id")}', YoutubeIE)
formats, subtitles = self._get_formats_and_subtitle(nextjs_json, display_id)
return {
'id': display_id,
'title': nextjs_json.get('title') or self._html_search_meta('og:title', webpage),
'formats': formats,
'subtitles': subtitles,
'description': nextjs_json.get('description') or None,
'duration': int_or_none(nextjs_json.get('duration')),
'thumbnails': [
{'url': thumbnail}
for thumbnail in traverse_obj(nextjs_json, ('default_thumbnails', ...))],
'channel_id': traverse_obj(nextjs_json, 'channel', 'channel_slug'),
'timestamp': parse_iso8601(nextjs_json.get('created')),
'release_timestamp': parse_iso8601(nextjs_json.get('published')),
'view_count': int_or_none(nextjs_json.get('views')),
}
+60
View File
@@ -0,0 +1,60 @@
from .common import InfoExtractor
from ..utils import int_or_none, merge_dicts
class AitubeKZVideoIE(InfoExtractor):
_VALID_URL = r'https?://aitube\.kz/(?:video|embed/)\?(?:[^\?]+)?id=(?P<id>[\w-]+)'
_TESTS = [{
# id paramater as first parameter
'url': 'https://aitube.kz/video?id=9291d29b-c038-49a1-ad42-3da2051d353c&playlistId=d55b1f5f-ef2a-4f23-b646-2a86275b86b7&season=1',
'info_dict': {
'id': '9291d29b-c038-49a1-ad42-3da2051d353c',
'ext': 'mp4',
'duration': 2174.0,
'channel_id': '94962f73-013b-432c-8853-1bd78ca860fe',
'like_count': int,
'channel': 'ASTANA TV',
'comment_count': int,
'view_count': int,
'description': 'Смотреть любимые сериалы и видео, поделиться видео и сериалами с друзьями и близкими',
'thumbnail': 'https://cdn.static02.aitube.kz/kz.aitudala.aitube.staticaccess/files/ddf2a2ff-bee3-409b-b5f2-2a8202bba75b',
'upload_date': '20221102',
'timestamp': 1667370519,
'title': 'Ангел хранитель 1 серия',
'channel_follower_count': int,
}
}, {
# embed url
'url': 'https://aitube.kz/embed/?id=9291d29b-c038-49a1-ad42-3da2051d353c',
'only_matching': True,
}, {
# id parameter is not as first paramater
'url': 'https://aitube.kz/video?season=1&id=9291d29b-c038-49a1-ad42-3da2051d353c&playlistId=d55b1f5f-ef2a-4f23-b646-2a86275b86b7',
'only_matching': True,
}]
def _real_extract(self, url):
video_id = self._match_id(url)
webpage = self._download_webpage(url, video_id)
nextjs_data = self._search_nextjs_data(webpage, video_id)['props']['pageProps']['videoInfo']
json_ld_data = self._search_json_ld(webpage, video_id)
formats, subtitles = self._extract_m3u8_formats_and_subtitles(
f'https://api-http.aitube.kz/kz.aitudala.aitube.staticaccess/video/{video_id}/video', video_id)
return merge_dicts({
'id': video_id,
'title': nextjs_data.get('title') or self._html_search_meta(['name', 'og:title'], webpage),
'description': nextjs_data.get('description'),
'formats': formats,
'subtitles': subtitles,
'view_count': (nextjs_data.get('viewCount')
or int_or_none(self._html_search_meta('ya:ovs:views_total', webpage))),
'like_count': nextjs_data.get('likeCount'),
'channel': nextjs_data.get('channelTitle'),
'channel_id': nextjs_data.get('channelId'),
'thumbnail': nextjs_data.get('coverUrl'),
'comment_count': nextjs_data.get('commentCount'),
'channel_follower_count': int_or_none(nextjs_data.get('channelSubscriberCount')),
}, json_ld_data)
-2
View File
@@ -112,8 +112,6 @@ class AllocineIE(InfoExtractor):
}) })
duration, view_count, timestamp = [None] * 3 duration, view_count, timestamp = [None] * 3
self._sort_formats(formats)
return { return {
'id': video_id, 'id': video_id,
'display_id': display_id, 'display_id': display_id,
-1
View File
@@ -22,7 +22,6 @@ class Alsace20TVBaseIE(InfoExtractor):
self._extract_smil_formats(fmt_url, video_id, fatal=False) self._extract_smil_formats(fmt_url, video_id, fatal=False)
if '/smil:_' in fmt_url if '/smil:_' in fmt_url
else self._extract_mpd_formats(fmt_url, video_id, mpd_id=res, fatal=False)) else self._extract_mpd_formats(fmt_url, video_id, mpd_id=res, fatal=False))
self._sort_formats(formats)
webpage = (url and self._download_webpage(url, video_id, fatal=False)) or '' webpage = (url and self._download_webpage(url, video_id, fatal=False)) or ''
thumbnail = url_or_none(dict_get(info, ('image', 'preview', )) or self._og_search_thumbnail(webpage)) thumbnail = url_or_none(dict_get(info, ('image', 'preview', )) or self._og_search_thumbnail(webpage))
+1 -3
View File
@@ -63,8 +63,6 @@ class AluraIE(InfoExtractor):
f['height'] = int('720' if m.group('res') == 'hd' else '480') f['height'] = int('720' if m.group('res') == 'hd' else '480')
formats.extend(video_format) formats.extend(video_format)
self._sort_formats(formats)
return { return {
'id': video_id, 'id': video_id,
'title': video_title, 'title': video_title,
@@ -113,7 +111,7 @@ class AluraIE(InfoExtractor):
raise ExtractorError('Unable to log in') raise ExtractorError('Unable to log in')
class AluraCourseIE(AluraIE): class AluraCourseIE(AluraIE): # XXX: Do not subclass from concrete IE
_VALID_URL = r'https?://(?:cursos\.)?alura\.com\.br/course/(?P<id>[^/]+)' _VALID_URL = r'https?://(?:cursos\.)?alura\.com\.br/course/(?P<id>[^/]+)'
_LOGIN_URL = 'https://cursos.alura.com.br/loginForm?urlAfterLogin=/loginForm' _LOGIN_URL = 'https://cursos.alura.com.br/loginForm?urlAfterLogin=/loginForm'
+109 -7
View File
@@ -1,5 +1,17 @@
import re
from .common import InfoExtractor from .common import InfoExtractor
from ..utils import ExtractorError, int_or_none from ..utils import (
ExtractorError,
clean_html,
float_or_none,
get_element_by_attribute,
get_element_by_class,
int_or_none,
js_to_json,
traverse_obj,
url_or_none,
)
class AmazonStoreIE(InfoExtractor): class AmazonStoreIE(InfoExtractor):
@@ -9,7 +21,7 @@ class AmazonStoreIE(InfoExtractor):
'url': 'https://www.amazon.co.uk/dp/B098XNCHLD/', 'url': 'https://www.amazon.co.uk/dp/B098XNCHLD/',
'info_dict': { 'info_dict': {
'id': 'B098XNCHLD', 'id': 'B098XNCHLD',
'title': 'md5:dae240564cbb2642170c02f7f0d7e472', 'title': str,
}, },
'playlist_mincount': 1, 'playlist_mincount': 1,
'playlist': [{ 'playlist': [{
@@ -20,28 +32,32 @@ class AmazonStoreIE(InfoExtractor):
'thumbnail': r're:^https?://.*\.jpg$', 'thumbnail': r're:^https?://.*\.jpg$',
'duration': 34, 'duration': 34,
}, },
}] }],
'expected_warnings': ['Unable to extract data'],
}, { }, {
'url': 'https://www.amazon.in/Sony-WH-1000XM4-Cancelling-Headphones-Bluetooth/dp/B0863TXGM3', 'url': 'https://www.amazon.in/Sony-WH-1000XM4-Cancelling-Headphones-Bluetooth/dp/B0863TXGM3',
'info_dict': { 'info_dict': {
'id': 'B0863TXGM3', 'id': 'B0863TXGM3',
'title': 'md5:d1d3352428f8f015706c84b31e132169', 'title': str,
}, },
'playlist_mincount': 4, 'playlist_mincount': 4,
'expected_warnings': ['Unable to extract data'],
}, { }, {
'url': 'https://www.amazon.com/dp/B0845NXCXF/', 'url': 'https://www.amazon.com/dp/B0845NXCXF/',
'info_dict': { 'info_dict': {
'id': 'B0845NXCXF', 'id': 'B0845NXCXF',
'title': 'md5:f3fa12779bf62ddb6a6ec86a360a858e', 'title': str,
}, },
'playlist-mincount': 1, 'playlist-mincount': 1,
'expected_warnings': ['Unable to extract data'],
}, { }, {
'url': 'https://www.amazon.es/Samsung-Smartphone-s-AMOLED-Quad-c%C3%A1mara-espa%C3%B1ola/dp/B08WX337PQ', 'url': 'https://www.amazon.es/Samsung-Smartphone-s-AMOLED-Quad-c%C3%A1mara-espa%C3%B1ola/dp/B08WX337PQ',
'info_dict': { 'info_dict': {
'id': 'B08WX337PQ', 'id': 'B08WX337PQ',
'title': 'md5:f3fa12779bf62ddb6a6ec86a360a858e', 'title': str,
}, },
'playlist_mincount': 1, 'playlist_mincount': 1,
'expected_warnings': ['Unable to extract data'],
}] }]
def _real_extract(self, url): def _real_extract(self, url):
@@ -52,7 +68,7 @@ class AmazonStoreIE(InfoExtractor):
try: try:
data_json = self._search_json( data_json = self._search_json(
r'var\s?obj\s?=\s?jQuery\.parseJSON\(\'', webpage, 'data', id, r'var\s?obj\s?=\s?jQuery\.parseJSON\(\'', webpage, 'data', id,
transform_source=lambda x: x.replace(R'\\u', R'\u')) transform_source=js_to_json)
except ExtractorError as e: except ExtractorError as e:
retry.error = e retry.error = e
@@ -66,3 +82,89 @@ class AmazonStoreIE(InfoExtractor):
'width': int_or_none(video.get('videoWidth')), 'width': int_or_none(video.get('videoWidth')),
} for video in (data_json.get('videos') or []) if video.get('isVideo') and video.get('url')] } for video in (data_json.get('videos') or []) if video.get('isVideo') and video.get('url')]
return self.playlist_result(entries, playlist_id=id, playlist_title=data_json.get('title')) return self.playlist_result(entries, playlist_id=id, playlist_title=data_json.get('title'))
class AmazonReviewsIE(InfoExtractor):
_VALID_URL = r'https?://(?:www\.)?amazon\.(?:[a-z]{2,3})(?:\.[a-z]{2})?/gp/customer-reviews/(?P<id>[^/&#$?]+)'
_TESTS = [{
'url': 'https://www.amazon.com/gp/customer-reviews/R10VE9VUSY19L3/ref=cm_cr_arp_d_rvw_ttl',
'info_dict': {
'id': 'R10VE9VUSY19L3',
'ext': 'mp4',
'title': 'Get squad #Suspicious',
'description': 'md5:7012695052f440a1e064e402d87e0afb',
'uploader': 'Kimberly Cronkright',
'average_rating': 1.0,
'thumbnail': r're:^https?://.*\.jpg$',
},
'expected_warnings': ['Review body was not found in webpage'],
}, {
'url': 'https://www.amazon.com/gp/customer-reviews/R10VE9VUSY19L3/ref=cm_cr_arp_d_rvw_ttl?language=es_US',
'info_dict': {
'id': 'R10VE9VUSY19L3',
'ext': 'mp4',
'title': 'Get squad #Suspicious',
'description': 'md5:7012695052f440a1e064e402d87e0afb',
'uploader': 'Kimberly Cronkright',
'average_rating': 1.0,
'thumbnail': r're:^https?://.*\.jpg$',
},
'expected_warnings': ['Review body was not found in webpage'],
}, {
'url': 'https://www.amazon.in/gp/customer-reviews/RV1CO8JN5VGXV/',
'info_dict': {
'id': 'RV1CO8JN5VGXV',
'ext': 'mp4',
'title': 'Not sure about its durability',
'description': 'md5:1a252c106357f0a3109ebf37d2e87494',
'uploader': 'Shoaib Gulzar',
'average_rating': 2.0,
'thumbnail': r're:^https?://.*\.jpg$',
},
'expected_warnings': ['Review body was not found in webpage'],
}]
def _real_extract(self, url):
video_id = self._match_id(url)
for retry in self.RetryManager():
webpage = self._download_webpage(url, video_id)
review_body = get_element_by_attribute('data-hook', 'review-body', webpage)
if not review_body:
retry.error = ExtractorError('Review body was not found in webpage', expected=True)
formats, subtitles = [], {}
manifest_url = self._search_regex(
r'data-video-url="([^"]+)"', review_body, 'm3u8 url', default=None)
if url_or_none(manifest_url):
fmts, subtitles = self._extract_m3u8_formats_and_subtitles(
manifest_url, video_id, 'mp4', fatal=False)
formats.extend(fmts)
video_url = self._search_regex(
r'<input[^>]+\bvalue="([^"]+)"[^>]+\bclass="video-url"', review_body, 'mp4 url', default=None)
if url_or_none(video_url):
formats.append({
'url': video_url,
'ext': 'mp4',
'format_id': 'http-mp4',
})
if not formats:
self.raise_no_formats('No video found for this customer review', expected=True)
return {
'id': video_id,
'title': (clean_html(get_element_by_attribute('data-hook', 'review-title', webpage))
or self._html_extract_title(webpage)),
'description': clean_html(traverse_obj(re.findall(
r'<span(?:\s+class="cr-original-review-content")?>(.+?)</span>', review_body), -1)),
'uploader': clean_html(get_element_by_class('a-profile-name', webpage)),
'average_rating': float_or_none(clean_html(get_element_by_attribute(
'data-hook', 'review-star-rating', webpage) or '').partition(' ')[0]),
'thumbnail': self._search_regex(
r'data-thumbnail-url="([^"]+)"', review_body, 'thumbnail', default=None),
'formats': formats,
'subtitles': subtitles,
}
+291
View File
@@ -0,0 +1,291 @@
import json
from .common import InfoExtractor
from ..utils import ExtractorError, int_or_none, traverse_obj, try_get
class AmazonMiniTVBaseIE(InfoExtractor):
def _real_initialize(self):
self._download_webpage(
'https://www.amazon.in/minitv', None,
note='Fetching guest session cookies')
AmazonMiniTVBaseIE.session_id = self._get_cookies('https://www.amazon.in')['session-id'].value
def _call_api(self, asin, data=None, note=None):
device = {'clientId': 'ATVIN', 'deviceLocale': 'en_GB'}
if data:
data['variables'].update({
'contentType': 'VOD',
'sessionIdToken': self.session_id,
**device,
})
resp = self._download_json(
f'https://www.amazon.in/minitv/api/web/{"graphql" if data else "prs"}',
asin, note=note, headers={'Content-Type': 'application/json'},
data=json.dumps(data).encode() if data else None,
query=None if data else {
'deviceType': 'A1WMMUXPCUJL4N',
'contentId': asin,
**device,
})
if resp.get('errors'):
raise ExtractorError(f'MiniTV said: {resp["errors"][0]["message"]}')
elif not data:
return resp
return resp['data'][data['operationName']]
class AmazonMiniTVIE(AmazonMiniTVBaseIE):
_VALID_URL = r'(?:https?://(?:www\.)?amazon\.in/minitv/tp/|amazonminitv:(?:amzn1\.dv\.gti\.)?)(?P<id>[a-f0-9-]+)'
_TESTS = [{
'url': 'https://www.amazon.in/minitv/tp/75fe3a75-b8fe-4499-8100-5c9424344840?referrer=https%3A%2F%2Fwww.amazon.in%2Fminitv',
'info_dict': {
'id': 'amzn1.dv.gti.75fe3a75-b8fe-4499-8100-5c9424344840',
'ext': 'mp4',
'title': 'May I Kiss You?',
'language': 'Hindi',
'thumbnail': r're:^https?://.*\.jpg$',
'description': 'md5:a549bfc747973e04feb707833474e59d',
'release_timestamp': 1644710400,
'release_date': '20220213',
'duration': 846,
'chapters': 'count:2',
'series': 'Couple Goals',
'series_id': 'amzn1.dv.gti.56521d46-b040-4fd5-872e-3e70476a04b0',
'season': 'Season 3',
'season_number': 3,
'season_id': 'amzn1.dv.gti.20331016-d9b9-4968-b991-c89fa4927a36',
'episode': 'May I Kiss You?',
'episode_number': 2,
'episode_id': 'amzn1.dv.gti.75fe3a75-b8fe-4499-8100-5c9424344840',
},
}, {
'url': 'https://www.amazon.in/minitv/tp/280d2564-584f-452f-9c98-7baf906e01ab?referrer=https%3A%2F%2Fwww.amazon.in%2Fminitv',
'info_dict': {
'id': 'amzn1.dv.gti.280d2564-584f-452f-9c98-7baf906e01ab',
'ext': 'mp4',
'title': 'Jahaan',
'language': 'Hindi',
'thumbnail': r're:^https?://.*\.jpg',
'description': 'md5:05eb765a77bf703f322f120ec6867339',
'release_timestamp': 1647475200,
'release_date': '20220317',
'duration': 783,
'chapters': [],
},
}, {
'url': 'https://www.amazon.in/minitv/tp/280d2564-584f-452f-9c98-7baf906e01ab',
'only_matching': True,
}, {
'url': 'amazonminitv:amzn1.dv.gti.280d2564-584f-452f-9c98-7baf906e01ab',
'only_matching': True,
}, {
'url': 'amazonminitv:280d2564-584f-452f-9c98-7baf906e01ab',
'only_matching': True,
}]
_GRAPHQL_QUERY_CONTENT = '''
query content($sessionIdToken: String!, $deviceLocale: String, $contentId: ID!, $contentType: ContentType!, $clientId: String) {
content(
applicationContextInput: {deviceLocale: $deviceLocale, sessionIdToken: $sessionIdToken, clientId: $clientId}
contentId: $contentId
contentType: $contentType
) {
contentId
name
... on Episode {
contentId
vodType
name
images
description {
synopsis
contentLengthInSeconds
}
publicReleaseDateUTC
audioTracks
seasonId
seriesId
seriesName
seasonNumber
episodeNumber
timecode {
endCreditsTime
}
}
... on MovieContent {
contentId
vodType
name
description {
synopsis
contentLengthInSeconds
}
images
publicReleaseDateUTC
audioTracks
}
}
}'''
def _real_extract(self, url):
asin = f'amzn1.dv.gti.{self._match_id(url)}'
prs = self._call_api(asin, note='Downloading playback info')
formats, subtitles = [], {}
for type_, asset in prs['playbackAssets'].items():
if not traverse_obj(asset, 'manifestUrl'):
continue
if type_ == 'hls':
m3u8_fmts, m3u8_subs = self._extract_m3u8_formats_and_subtitles(
asset['manifestUrl'], asin, ext='mp4', entry_protocol='m3u8_native',
m3u8_id=type_, fatal=False)
formats.extend(m3u8_fmts)
subtitles = self._merge_subtitles(subtitles, m3u8_subs)
elif type_ == 'dash':
mpd_fmts, mpd_subs = self._extract_mpd_formats_and_subtitles(
asset['manifestUrl'], asin, mpd_id=type_, fatal=False)
formats.extend(mpd_fmts)
subtitles = self._merge_subtitles(subtitles, mpd_subs)
else:
self.report_warning(f'Unknown asset type: {type_}')
title_info = self._call_api(
asin, note='Downloading title info', data={
'operationName': 'content',
'variables': {'contentId': asin},
'query': self._GRAPHQL_QUERY_CONTENT,
})
credits_time = try_get(title_info, lambda x: x['timecode']['endCreditsTime'] / 1000)
is_episode = title_info.get('vodType') == 'EPISODE'
return {
'id': asin,
'title': title_info.get('name'),
'formats': formats,
'subtitles': subtitles,
'language': traverse_obj(title_info, ('audioTracks', 0)),
'thumbnails': [{
'id': type_,
'url': url,
} for type_, url in (title_info.get('images') or {}).items()],
'description': traverse_obj(title_info, ('description', 'synopsis')),
'release_timestamp': int_or_none(try_get(title_info, lambda x: x['publicReleaseDateUTC'] / 1000)),
'duration': traverse_obj(title_info, ('description', 'contentLengthInSeconds')),
'chapters': [{
'start_time': credits_time,
'title': 'End Credits',
}] if credits_time else [],
'series': title_info.get('seriesName'),
'series_id': title_info.get('seriesId'),
'season_number': title_info.get('seasonNumber'),
'season_id': title_info.get('seasonId'),
'episode': title_info.get('name') if is_episode else None,
'episode_number': title_info.get('episodeNumber'),
'episode_id': asin if is_episode else None,
}
class AmazonMiniTVSeasonIE(AmazonMiniTVBaseIE):
IE_NAME = 'amazonminitv:season'
_VALID_URL = r'amazonminitv:season:(?:amzn1\.dv\.gti\.)?(?P<id>[a-f0-9-]+)'
IE_DESC = 'Amazon MiniTV Season, "minitv:season:" prefix'
_TESTS = [{
'url': 'amazonminitv:season:amzn1.dv.gti.0aa996eb-6a1b-4886-a342-387fbd2f1db0',
'playlist_mincount': 6,
'info_dict': {
'id': 'amzn1.dv.gti.0aa996eb-6a1b-4886-a342-387fbd2f1db0',
},
}, {
'url': 'amazonminitv:season:0aa996eb-6a1b-4886-a342-387fbd2f1db0',
'only_matching': True,
}]
_GRAPHQL_QUERY = '''
query getEpisodes($sessionIdToken: String!, $clientId: String, $episodeOrSeasonId: ID!, $deviceLocale: String) {
getEpisodes(
applicationContextInput: {sessionIdToken: $sessionIdToken, deviceLocale: $deviceLocale, clientId: $clientId}
episodeOrSeasonId: $episodeOrSeasonId
) {
episodes {
... on Episode {
contentId
name
images
seriesName
seasonId
seriesId
seasonNumber
episodeNumber
description {
synopsis
contentLengthInSeconds
}
publicReleaseDateUTC
}
}
}
}
'''
def _entries(self, asin):
season_info = self._call_api(
asin, note='Downloading season info', data={
'operationName': 'getEpisodes',
'variables': {'episodeOrSeasonId': asin},
'query': self._GRAPHQL_QUERY,
})
for episode in season_info['episodes']:
yield self.url_result(
f'amazonminitv:{episode["contentId"]}', AmazonMiniTVIE, episode['contentId'])
def _real_extract(self, url):
asin = f'amzn1.dv.gti.{self._match_id(url)}'
return self.playlist_result(self._entries(asin), asin)
class AmazonMiniTVSeriesIE(AmazonMiniTVBaseIE):
IE_NAME = 'amazonminitv:series'
_VALID_URL = r'amazonminitv:series:(?:amzn1\.dv\.gti\.)?(?P<id>[a-f0-9-]+)'
IE_DESC = 'Amazon MiniTV Series, "minitv:series:" prefix'
_TESTS = [{
'url': 'amazonminitv:series:amzn1.dv.gti.56521d46-b040-4fd5-872e-3e70476a04b0',
'playlist_mincount': 3,
'info_dict': {
'id': 'amzn1.dv.gti.56521d46-b040-4fd5-872e-3e70476a04b0',
},
}, {
'url': 'amazonminitv:series:56521d46-b040-4fd5-872e-3e70476a04b0',
'only_matching': True,
}]
_GRAPHQL_QUERY = '''
query getSeasons($sessionIdToken: String!, $deviceLocale: String, $episodeOrSeasonOrSeriesId: ID!, $clientId: String) {
getSeasons(
applicationContextInput: {deviceLocale: $deviceLocale, sessionIdToken: $sessionIdToken, clientId: $clientId}
episodeOrSeasonOrSeriesId: $episodeOrSeasonOrSeriesId
) {
seasons {
seasonId
}
}
}
'''
def _entries(self, asin):
season_info = self._call_api(
asin, note='Downloading series info', data={
'operationName': 'getSeasons',
'variables': {'episodeOrSeasonOrSeriesId': asin},
'query': self._GRAPHQL_QUERY,
})
for season in season_info['seasons']:
yield self.url_result(f'amazonminitv:season:{season["seasonId"]}', AmazonMiniTVSeasonIE, season['seasonId'])
def _real_extract(self, url):
asin = f'amzn1.dv.gti.{self._match_id(url)}'
return self.playlist_result(self._entries(asin), asin)
+1 -2
View File
@@ -9,7 +9,7 @@ from ..utils import (
) )
class AMCNetworksIE(ThePlatformIE): class AMCNetworksIE(ThePlatformIE): # XXX: Do not subclass from concrete IE
_VALID_URL = r'https?://(?:www\.)?(?P<site>amc|bbcamerica|ifc|(?:we|sundance)tv)\.com/(?P<id>(?:movies|shows(?:/[^/]+)+)/[^/?#&]+)' _VALID_URL = r'https?://(?:www\.)?(?P<site>amc|bbcamerica|ifc|(?:we|sundance)tv)\.com/(?P<id>(?:movies|shows(?:/[^/]+)+)/[^/?#&]+)'
_TESTS = [{ _TESTS = [{
'url': 'https://www.bbcamerica.com/shows/the-graham-norton-show/videos/tina-feys-adorable-airline-themed-family-dinner--51631', 'url': 'https://www.bbcamerica.com/shows/the-graham-norton-show/videos/tina-feys-adorable-airline-themed-family-dinner--51631',
@@ -106,7 +106,6 @@ class AMCNetworksIE(ThePlatformIE):
media_url = update_url_query(media_url, query) media_url = update_url_query(media_url, query)
formats, subtitles = self._extract_theplatform_smil( formats, subtitles = self._extract_theplatform_smil(
media_url, video_id) media_url, video_id)
self._sort_formats(formats)
thumbnails = [] thumbnails = []
thumbnail_urls = [properties.get('imageDesktop')] thumbnail_urls = [properties.get('imageDesktop')]
+64 -14
View File
@@ -11,7 +11,7 @@ from ..utils import (
class AmericasTestKitchenIE(InfoExtractor): class AmericasTestKitchenIE(InfoExtractor):
_VALID_URL = r'https?://(?:www\.)?americastestkitchen\.com/(?:cooks(?:country|illustrated)/)?(?P<resource_type>episode|videos)/(?P<id>\d+)' _VALID_URL = r'https?://(?:www\.)?(?:americastestkitchen|cooks(?:country|illustrated))\.com/(?:cooks(?:country|illustrated)/)?(?P<resource_type>episode|videos)/(?P<id>\d+)'
_TESTS = [{ _TESTS = [{
'url': 'https://www.americastestkitchen.com/episode/582-weeknight-japanese-suppers', 'url': 'https://www.americastestkitchen.com/episode/582-weeknight-japanese-suppers',
'md5': 'b861c3e365ac38ad319cfd509c30577f', 'md5': 'b861c3e365ac38ad319cfd509c30577f',
@@ -72,6 +72,12 @@ class AmericasTestKitchenIE(InfoExtractor):
}, { }, {
'url': 'https://www.americastestkitchen.com/cooksillustrated/videos/4478-beef-wellington', 'url': 'https://www.americastestkitchen.com/cooksillustrated/videos/4478-beef-wellington',
'only_matching': True, 'only_matching': True,
}, {
'url': 'https://www.cookscountry.com/episode/564-when-only-chocolate-will-do',
'only_matching': True,
}, {
'url': 'https://www.cooksillustrated.com/videos/4478-beef-wellington',
'only_matching': True,
}] }]
def _real_extract(self, url): def _real_extract(self, url):
@@ -100,7 +106,7 @@ class AmericasTestKitchenIE(InfoExtractor):
class AmericasTestKitchenSeasonIE(InfoExtractor): class AmericasTestKitchenSeasonIE(InfoExtractor):
_VALID_URL = r'https?://(?:www\.)?americastestkitchen\.com(?P<show>/cookscountry)?/episodes/browse/season_(?P<id>\d+)' _VALID_URL = r'https?://(?:www\.)?(?P<show>americastestkitchen|(?P<cooks>cooks(?:country|illustrated)))\.com(?:(?:/(?P<show2>cooks(?:country|illustrated)))?(?:/?$|(?<!ated)(?<!ated\.com)/episodes/browse/season_(?P<season>\d+)))'
_TESTS = [{ _TESTS = [{
# ATK Season # ATK Season
'url': 'https://www.americastestkitchen.com/episodes/browse/season_1', 'url': 'https://www.americastestkitchen.com/episodes/browse/season_1',
@@ -117,29 +123,73 @@ class AmericasTestKitchenSeasonIE(InfoExtractor):
'title': 'Season 12', 'title': 'Season 12',
}, },
'playlist_count': 13, 'playlist_count': 13,
}, {
# America's Test Kitchen Series
'url': 'https://www.americastestkitchen.com/',
'info_dict': {
'id': 'americastestkitchen',
'title': 'America\'s Test Kitchen',
},
'playlist_count': 558,
}, {
# Cooks Country Series
'url': 'https://www.americastestkitchen.com/cookscountry',
'info_dict': {
'id': 'cookscountry',
'title': 'Cook\'s Country',
},
'playlist_count': 199,
}, {
'url': 'https://www.americastestkitchen.com/cookscountry/',
'only_matching': True,
}, {
'url': 'https://www.cookscountry.com/episodes/browse/season_12',
'only_matching': True,
}, {
'url': 'https://www.cookscountry.com',
'only_matching': True,
}, {
'url': 'https://www.americastestkitchen.com/cooksillustrated/',
'only_matching': True,
}, {
'url': 'https://www.cooksillustrated.com',
'only_matching': True,
}] }]
def _real_extract(self, url): def _real_extract(self, url):
show_path, season_number = self._match_valid_url(url).group('show', 'id') season_number, show1, show = self._match_valid_url(url).group('season', 'show', 'show2')
season_number = int(season_number) show_path = ('/' + show) if show else ''
show = show or show1
season_number = int_or_none(season_number)
slug = 'cco' if show_path == '/cookscountry' else 'atk' slug, title = {
'americastestkitchen': ('atk', 'America\'s Test Kitchen'),
'cookscountry': ('cco', 'Cook\'s Country'),
'cooksillustrated': ('cio', 'Cook\'s Illustrated'),
}[show]
season = 'Season %d' % season_number facet_filters = [
'search_document_klass:episode',
'search_show_slug:' + slug,
]
if season_number:
playlist_id = 'season_%d' % season_number
playlist_title = 'Season %d' % season_number
facet_filters.append('search_season_list:' + playlist_title)
else:
playlist_id = show
playlist_title = title
season_search = self._download_json( season_search = self._download_json(
'https://y1fnzxui30-dsn.algolia.net/1/indexes/everest_search_%s_season_desc_production' % slug, 'https://y1fnzxui30-dsn.algolia.net/1/indexes/everest_search_%s_season_desc_production' % slug,
season, headers={ playlist_id, headers={
'Origin': 'https://www.americastestkitchen.com', 'Origin': 'https://www.americastestkitchen.com',
'X-Algolia-API-Key': '8d504d0099ed27c1b73708d22871d805', 'X-Algolia-API-Key': '8d504d0099ed27c1b73708d22871d805',
'X-Algolia-Application-Id': 'Y1FNZXUI30', 'X-Algolia-Application-Id': 'Y1FNZXUI30',
}, query={ }, query={
'facetFilters': json.dumps([ 'facetFilters': json.dumps(facet_filters),
'search_season_list:' + season, 'attributesToRetrieve': 'description,search_%s_episode_number,search_document_date,search_url,title,search_atk_episode_season' % slug,
'search_document_klass:episode',
'search_show_slug:' + slug,
]),
'attributesToRetrieve': 'description,search_%s_episode_number,search_document_date,search_url,title' % slug,
'attributesToHighlight': '', 'attributesToHighlight': '',
'hitsPerPage': 1000, 'hitsPerPage': 1000,
}) })
@@ -162,4 +212,4 @@ class AmericasTestKitchenSeasonIE(InfoExtractor):
} }
return self.playlist_result( return self.playlist_result(
entries(), 'season_%d' % season_number, season) entries(), playlist_id, playlist_title)
+1 -3
View File
@@ -10,7 +10,7 @@ from ..utils import (
) )
class AMPIE(InfoExtractor): class AMPIE(InfoExtractor): # XXX: Conventionally, base classes should end with BaseIE/InfoExtractor
# parse Akamai Adaptive Media Player feed # parse Akamai Adaptive Media Player feed
def _extract_feed_info(self, url): def _extract_feed_info(self, url):
feed = self._download_json( feed = self._download_json(
@@ -84,8 +84,6 @@ class AMPIE(InfoExtractor):
'ext': ext, 'ext': ext,
}) })
self._sort_formats(formats)
timestamp = unified_timestamp(item.get('pubDate'), ' ') or parse_iso8601(item.get('dc-date')) timestamp = unified_timestamp(item.get('pubDate'), ' ') or parse_iso8601(item.get('dc-date'))
return { return {
+98
View File
@@ -0,0 +1,98 @@
from .common import InfoExtractor
from ..utils import (
clean_html,
float_or_none,
int_or_none,
str_or_none,
traverse_obj,
unified_timestamp
)
class AnchorFMEpisodeIE(InfoExtractor):
_VALID_URL = r'https?://anchor\.fm/(?P<channel_name>\w+)/(?:embed/)?episodes/[\w-]+-(?P<episode_id>\w+)'
_EMBED_REGEX = [rf'<iframe[^>]+\bsrc=[\'"](?P<url>{_VALID_URL})']
_TESTS = [{
'url': 'https://anchor.fm/lovelyti/episodes/Chrisean-Rock-takes-to-twitter-to-announce-shes-pregnant--Blueface-denies-he-is-the-father-e1tpt3d',
'info_dict': {
'id': 'e1tpt3d',
'ext': 'mp3',
'title': ' Chrisean Rock takes to twitter to announce she\'s pregnant, Blueface denies he is the father!',
'description': 'md5:207d167de3e28ceb4ddc1ebf5a30044c',
'thumbnail': 'https://s3-us-west-2.amazonaws.com/anchor-generated-image-bank/production/podcast_uploaded_nologo/1034827/1034827-1658438968460-5f3bfdf3601e8.jpg',
'duration': 624.718,
'uploader': 'Lovelyti ',
'uploader_id': '991541',
'channel': 'lovelyti',
'modified_date': '20230121',
'modified_timestamp': 1674285178,
'release_date': '20230121',
'release_timestamp': 1674285179,
'episode_id': 'e1tpt3d',
}
}, {
# embed url
'url': 'https://anchor.fm/apakatatempo/embed/episodes/S2E75-Perang-Bintang-di-Balik-Kasus-Ferdy-Sambo-dan-Ismail-Bolong-e1shjqd',
'info_dict': {
'id': 'e1shjqd',
'ext': 'mp3',
'title': 'S2E75 Perang Bintang di Balik Kasus Ferdy Sambo dan Ismail Bolong',
'description': 'md5:9e95ad9293bf00178bf8d33e9cb92c41',
'duration': 1042.008,
'thumbnail': 'https://s3-us-west-2.amazonaws.com/anchor-generated-image-bank/production/podcast_uploaded_episode400/2627805/2627805-1671590688729-4db3882ac9e4b.jpg',
'release_date': '20221221',
'release_timestamp': 1671595916,
'modified_date': '20221221',
'modified_timestamp': 1671590834,
'channel': 'apakatatempo',
'uploader': 'Podcast Tempo',
'uploader_id': '2585461',
'season': 'Season 2',
'season_number': 2,
'episode_id': 'e1shjqd',
}
}]
_WEBPAGE_TESTS = [{
'url': 'https://podcast.tempo.co/podcast/192/perang-bintang-di-balik-kasus-ferdy-sambo-dan-ismail-bolong',
'info_dict': {
'id': 'e1shjqd',
'ext': 'mp3',
'release_date': '20221221',
'duration': 1042.008,
'season': 'Season 2',
'modified_timestamp': 1671590834,
'uploader_id': '2585461',
'modified_date': '20221221',
'description': 'md5:9e95ad9293bf00178bf8d33e9cb92c41',
'season_number': 2,
'title': 'S2E75 Perang Bintang di Balik Kasus Ferdy Sambo dan Ismail Bolong',
'release_timestamp': 1671595916,
'episode_id': 'e1shjqd',
'thumbnail': 'https://s3-us-west-2.amazonaws.com/anchor-generated-image-bank/production/podcast_uploaded_episode400/2627805/2627805-1671590688729-4db3882ac9e4b.jpg',
'uploader': 'Podcast Tempo',
'channel': 'apakatatempo',
}
}]
def _real_extract(self, url):
channel_name, episode_id = self._match_valid_url(url).group('channel_name', 'episode_id')
api_data = self._download_json(f'https://anchor.fm/api/v3/episodes/{episode_id}', episode_id)
return {
'id': episode_id,
'title': traverse_obj(api_data, ('episode', 'title')),
'url': traverse_obj(api_data, ('episode', 'episodeEnclosureUrl'), ('episodeAudios', 0, 'url')),
'ext': 'mp3',
'vcodec': 'none',
'thumbnail': traverse_obj(api_data, ('episode', 'episodeImage')),
'description': clean_html(traverse_obj(api_data, ('episode', ('description', 'descriptionPreview')), get_all=False)),
'duration': float_or_none(traverse_obj(api_data, ('episode', 'duration')), 1000),
'modified_timestamp': unified_timestamp(traverse_obj(api_data, ('episode', 'modified'))),
'release_timestamp': int_or_none(traverse_obj(api_data, ('episode', 'publishOnUnixTimestamp'))),
'episode_id': episode_id,
'uploader': traverse_obj(api_data, ('creator', 'name')),
'uploader_id': str_or_none(traverse_obj(api_data, ('creator', 'userId'))),
'season_number': int_or_none(traverse_obj(api_data, ('episode', 'podcastSeasonNumber'))),
'channel': channel_name or traverse_obj(api_data, ('creator', 'vanitySlug')),
}
-1
View File
@@ -19,7 +19,6 @@ class Ant1NewsGrBaseIE(InfoExtractor):
raise ExtractorError('no source found for %s' % video_id) raise ExtractorError('no source found for %s' % video_id)
formats, subs = (self._extract_m3u8_formats_and_subtitles(source, video_id, 'mp4') formats, subs = (self._extract_m3u8_formats_and_subtitles(source, video_id, 'mp4')
if determine_ext(source) == 'm3u8' else ([{'url': source}], {})) if determine_ext(source) == 'm3u8' else ([{'url': source}], {}))
self._sort_formats(formats)
thumbnails = scale_thumbnails_to_max_format_width( thumbnails = scale_thumbnails_to_max_format_width(
formats, [{'url': info['thumb']}], r'(?<=/imgHandler/)\d+') formats, [{'url': info['thumb']}], r'(?<=/imgHandler/)\d+')
return { return {
-2
View File
@@ -354,8 +354,6 @@ class AnvatoIE(InfoExtractor):
}) })
formats.append(a_format) formats.append(a_format)
self._sort_formats(formats)
subtitles = {} subtitles = {}
for caption in video_data.get('captions', []): for caption in video_data.get('captions', []):
a_caption = { a_caption = {
+1 -2
View File
@@ -9,7 +9,7 @@ from ..utils import (
) )
class AolIE(YahooIE): class AolIE(YahooIE): # XXX: Do not subclass from concrete IE
IE_NAME = 'aol.com' IE_NAME = 'aol.com'
_VALID_URL = r'(?:aol-video:|https?://(?:www\.)?aol\.(?:com|ca|co\.uk|de|jp)/video/(?:[^/]+/)*)(?P<id>\d{9}|[0-9a-f]{24}|[0-9a-f]{8}-(?:[0-9a-f]{4}-){3}[0-9a-f]{12})' _VALID_URL = r'(?:aol-video:|https?://(?:www\.)?aol\.(?:com|ca|co\.uk|de|jp)/video/(?:[^/]+/)*)(?P<id>\d{9}|[0-9a-f]{24}|[0-9a-f]{8}-(?:[0-9a-f]{4}-){3}[0-9a-f]{12})'
@@ -119,7 +119,6 @@ class AolIE(YahooIE):
'height': int_or_none(qs.get('h', [None])[0]), 'height': int_or_none(qs.get('h', [None])[0]),
}) })
formats.append(f) formats.append(f)
self._sort_formats(formats)
return { return {
'id': video_id, 'id': video_id,
-1
View File
@@ -72,7 +72,6 @@ class APAIE(InfoExtractor):
'format_id': format_id, 'format_id': format_id,
'height': height, 'height': height,
}) })
self._sort_formats(formats)
return { return {
'id': video_id, 'id': video_id,
-1
View File
@@ -73,7 +73,6 @@ class AparatIE(InfoExtractor):
r'(\d+)[pP]', label or '', 'height', r'(\d+)[pP]', label or '', 'height',
default=None)), default=None)),
}) })
self._sort_formats(formats)
info = self._search_json_ld(webpage, video_id, default={}) info = self._search_json_ld(webpage, video_id, default={})
-3
View File
@@ -120,7 +120,6 @@ class AppleTrailersIE(InfoExtractor):
'height': int_or_none(size_data.get('height')), 'height': int_or_none(size_data.get('height')),
'language': version[:2], 'language': version[:2],
}) })
self._sort_formats(formats)
entries.append({ entries.append({
'id': movie + '-' + re.sub(r'[^a-zA-Z0-9]', '', clip_title).lower(), 'id': movie + '-' + re.sub(r'[^a-zA-Z0-9]', '', clip_title).lower(),
@@ -185,8 +184,6 @@ class AppleTrailersIE(InfoExtractor):
'height': int_or_none(format['height']), 'height': int_or_none(format['height']),
}) })
self._sort_formats(formats)
playlist.append({ playlist.append({
'_type': 'video', '_type': 'video',
'id': video_id, 'id': video_id,
+237 -1
View File
@@ -1,8 +1,10 @@
import json import json
import re import re
import urllib.error
import urllib.parse import urllib.parse
from .common import InfoExtractor from .common import InfoExtractor
from .naver import NaverBaseIE
from .youtube import YoutubeBaseInfoExtractor, YoutubeIE from .youtube import YoutubeBaseInfoExtractor, YoutubeIE
from ..compat import compat_HTTPError, compat_urllib_parse_unquote from ..compat import compat_HTTPError, compat_urllib_parse_unquote
from ..utils import ( from ..utils import (
@@ -312,7 +314,7 @@ class ArchiveOrgIE(InfoExtractor):
}) })
for entry in entries.values(): for entry in entries.values():
self._sort_formats(entry['formats'], ('source', )) entry['_format_sort_fields'] = ('source', )
if len(entries) == 1: if len(entries) == 1:
# If there's only one item, use it as the main info dict # If there's only one item, use it as the main info dict
@@ -945,3 +947,237 @@ class YoutubeWebArchiveIE(InfoExtractor):
if not info.get('title'): if not info.get('title'):
info['title'] = video_id info['title'] = video_id
return info return info
class VLiveWebArchiveIE(InfoExtractor):
IE_NAME = 'web.archive:vlive'
IE_DESC = 'web.archive.org saved vlive videos'
_VALID_URL = r'''(?x)
(?:https?://)?web\.archive\.org/
(?:web/)?(?:(?P<date>[0-9]{14})?[0-9A-Za-z_*]*/)? # /web and the version index is optional
(?:https?(?::|%3[Aa])//)?(?:
(?:(?:www|m)\.)?vlive\.tv(?::(?:80|443))?/(?:video|embed)/(?P<id>[0-9]+) # VLive URL
)
'''
_TESTS = [{
'url': 'https://web.archive.org/web/20221221144331/http://www.vlive.tv/video/1326',
'md5': 'cc7314812855ce56de70a06a27314983',
'info_dict': {
'id': '1326',
'ext': 'mp4',
'title': "Girl's Day's Broadcast",
'creator': "Girl's Day",
'view_count': int,
'uploader_id': 'muploader_a',
'uploader_url': None,
'uploader': None,
'upload_date': '20150817',
'thumbnail': r're:^https?://.*\.(?:jpg|png)$',
'timestamp': 1439816449,
'like_count': int,
'channel': 'Girl\'s Day',
'channel_id': 'FDF27',
'comment_count': int,
'release_timestamp': 1439818140,
'release_date': '20150817',
'duration': 1014,
},
'params': {
'skip_download': True,
},
}, {
'url': 'https://web.archive.org/web/20221221182103/http://www.vlive.tv/video/16937',
'info_dict': {
'id': '16937',
'ext': 'mp4',
'title': '첸백시 걍방',
'creator': 'EXO',
'view_count': int,
'subtitles': 'mincount:12',
'uploader_id': 'muploader_j',
'uploader_url': 'http://vlive.tv',
'uploader': None,
'upload_date': '20161112',
'thumbnail': r're:^https?://.*\.(?:jpg|png)$',
'timestamp': 1478923074,
'like_count': int,
'channel': 'EXO',
'channel_id': 'F94BD',
'comment_count': int,
'release_timestamp': 1478924280,
'release_date': '20161112',
'duration': 906,
},
'params': {
'skip_download': True,
},
}, {
'url': 'https://web.archive.org/web/20221127190050/http://www.vlive.tv/video/101870',
'info_dict': {
'id': '101870',
'ext': 'mp4',
'title': '[ⓓ xV] “레벨이들 매력에 반해? 안 반해?” 움직이는 HD 포토 (레드벨벳:Red Velvet)',
'creator': 'Dispatch',
'view_count': int,
'subtitles': 'mincount:6',
'uploader_id': 'V__FRA08071',
'uploader_url': 'http://vlive.tv',
'uploader': None,
'upload_date': '20181130',
'thumbnail': r're:^https?://.*\.(?:jpg|png)$',
'timestamp': 1543601327,
'like_count': int,
'channel': 'Dispatch',
'channel_id': 'C796F3',
'comment_count': int,
'release_timestamp': 1543601040,
'release_date': '20181130',
'duration': 279,
},
'params': {
'skip_download': True,
},
}]
# The wayback machine has special timestamp and "mode" values:
# timestamp:
# 1 = the first capture
# 2 = the last capture
# mode:
# id_ = Identity - perform no alterations of the original resource, return it as it was archived.
_WAYBACK_BASE_URL = 'https://web.archive.org/web/2id_/'
def _download_archived_page(self, url, video_id, *, timestamp='2', **kwargs):
for retry in self.RetryManager():
try:
return self._download_webpage(f'https://web.archive.org/web/{timestamp}id_/{url}', video_id, **kwargs)
except ExtractorError as e:
if isinstance(e.cause, urllib.error.HTTPError) and e.cause.code == 404:
raise ExtractorError('Page was not archived', expected=True)
retry.error = e
continue
def _download_archived_json(self, url, video_id, **kwargs):
page = self._download_archived_page(url, video_id, **kwargs)
if not page:
raise ExtractorError('Page was not archived', expected=True)
else:
return self._parse_json(page, video_id)
def _extract_formats_from_m3u8(self, m3u8_url, params, video_id):
m3u8_doc = self._download_archived_page(m3u8_url, video_id, note='Downloading m3u8', query=params, fatal=False)
if not m3u8_doc:
return
# M3U8 document should be changed to archive domain
m3u8_doc = m3u8_doc.splitlines()
url_base = m3u8_url.rsplit('/', 1)[0]
first_segment = None
for i, line in enumerate(m3u8_doc):
if not line.startswith('#'):
m3u8_doc[i] = f'{self._WAYBACK_BASE_URL}{url_base}/{line}?{urllib.parse.urlencode(params)}'
first_segment = first_segment or m3u8_doc[i]
# Segments may not have been archived. See https://web.archive.org/web/20221127190050/http://www.vlive.tv/video/101870
urlh = self._request_webpage(HEADRequest(first_segment), video_id, errnote=False,
fatal=False, note='Check first segment availablity')
if urlh:
formats, subtitles = self._parse_m3u8_formats_and_subtitles('\n'.join(m3u8_doc), ext='mp4', video_id=video_id)
if subtitles:
self._report_ignoring_subs('m3u8')
return formats
# Closely follows the logic of the ArchiveTeam grab script
# See: https://github.com/ArchiveTeam/vlive-grab/blob/master/vlive.lua
def _real_extract(self, url):
video_id, url_date = self._match_valid_url(url).group('id', 'date')
webpage = self._download_archived_page(f'https://www.vlive.tv/video/{video_id}', video_id, timestamp=url_date)
player_info = self._search_json(r'__PRELOADED_STATE__\s*=', webpage, 'player info', video_id)
user_country = traverse_obj(player_info, ('common', 'userCountry'))
main_script_url = self._search_regex(r'<script\s+src="([^"]+/js/main\.[^"]+\.js)"', webpage, 'main script url')
main_script = self._download_archived_page(main_script_url, video_id, note='Downloading main script')
app_id = self._search_regex(r'appId\s*=\s*"([^"]+)"', main_script, 'app id')
inkey = self._download_archived_json(
f'https://www.vlive.tv/globalv-web/vam-web/video/v1.0/vod/{video_id}/inkey', video_id, note='Fetching inkey', query={
'appId': app_id,
'platformType': 'PC',
'gcc': user_country,
'locale': 'en_US',
}, fatal=False)
vod_id = traverse_obj(player_info, ('postDetail', 'post', 'officialVideo', 'vodId'))
vod_data = self._download_archived_json(
f'https://apis.naver.com/rmcnmv/rmcnmv/vod/play/v2.0/{vod_id}', video_id, note='Fetching vod data', query={
'key': inkey.get('inkey'),
'pid': 'rmcPlayer_16692457559726800', # partially unix time and partially random. Fixed value used by archiveteam project
'sid': '2024',
'ver': '2.0',
'devt': 'html5_pc',
'doct': 'json',
'ptc': 'https',
'sptc': 'https',
'cpt': 'vtt',
'ctls': '%7B%22visible%22%3A%7B%22fullscreen%22%3Atrue%2C%22logo%22%3Afalse%2C%22playbackRate%22%3Afalse%2C%22scrap%22%3Afalse%2C%22playCount%22%3Atrue%2C%22commentCount%22%3Atrue%2C%22title%22%3Atrue%2C%22writer%22%3Atrue%2C%22expand%22%3Afalse%2C%22subtitles%22%3Atrue%2C%22thumbnails%22%3Atrue%2C%22quality%22%3Atrue%2C%22setting%22%3Atrue%2C%22script%22%3Afalse%2C%22logoDimmed%22%3Atrue%2C%22badge%22%3Atrue%2C%22seekingTime%22%3Atrue%2C%22muted%22%3Atrue%2C%22muteButton%22%3Afalse%2C%22viewerNotice%22%3Afalse%2C%22linkCount%22%3Afalse%2C%22createTime%22%3Afalse%2C%22thumbnail%22%3Atrue%7D%2C%22clicked%22%3A%7B%22expand%22%3Afalse%2C%22subtitles%22%3Afalse%7D%7D',
'pv': '4.26.9',
'dr': '1920x1080',
'cpl': 'en_US',
'lc': 'en_US',
'adi': '%5B%7B%22type%22%3A%22pre%22%2C%22exposure%22%3Afalse%2C%22replayExposure%22%3Afalse%7D%5D',
'adu': '%2F',
'videoId': vod_id,
'cc': user_country,
})
formats = []
streams = traverse_obj(vod_data, ('streams', ...))
if len(streams) > 1:
self.report_warning('Multiple streams found. Only the first stream will be downloaded.')
stream = streams[0]
max_stream = max(
stream.get('videos') or [],
key=lambda v: traverse_obj(v, ('bitrate', 'video'), default=0), default=None)
if max_stream is not None:
params = {arg.get('name'): arg.get('value') for arg in stream.get('keys', []) if arg.get('type') == 'param'}
formats = self._extract_formats_from_m3u8(max_stream.get('source'), params, video_id) or []
# For parts of the project MP4 files were archived
max_video = max(
traverse_obj(vod_data, ('videos', 'list', ...)),
key=lambda v: traverse_obj(v, ('bitrate', 'video'), default=0), default=None)
if max_video is not None:
video_url = self._WAYBACK_BASE_URL + max_video.get('source')
urlh = self._request_webpage(HEADRequest(video_url), video_id, errnote=False,
fatal=False, note='Check video availablity')
if urlh:
formats.append({'url': video_url})
return {
'id': video_id,
'formats': formats,
**traverse_obj(player_info, ('postDetail', 'post', {
'title': ('officialVideo', 'title', {str}),
'creator': ('author', 'nickname', {str}),
'channel': ('channel', 'channelName', {str}),
'channel_id': ('channel', 'channelCode', {str}),
'duration': ('officialVideo', 'playTime', {int_or_none}),
'view_count': ('officialVideo', 'playCount', {int_or_none}),
'like_count': ('officialVideo', 'likeCount', {int_or_none}),
'comment_count': ('officialVideo', 'commentCount', {int_or_none}),
'timestamp': ('officialVideo', 'createdAt', {lambda x: int_or_none(x, scale=1000)}),
'release_timestamp': ('officialVideo', 'willStartAt', {lambda x: int_or_none(x, scale=1000)}),
})),
**traverse_obj(vod_data, ('meta', {
'uploader_id': ('user', 'id', {str}),
'uploader': ('user', 'name', {str}),
'uploader_url': ('user', 'url', {url_or_none}),
'thumbnail': ('cover', 'source', {url_or_none}),
}), expected_type=lambda x: x or None),
**NaverBaseIE.process_subtitles(vod_data, lambda x: [self._WAYBACK_BASE_URL + x]),
}
-1
View File
@@ -144,7 +144,6 @@ class ArcPublishingIE(InfoExtractor):
'url': s_url, 'url': s_url,
'quality': -10, 'quality': -10,
}) })
self._sort_formats(formats)
subtitles = {} subtitles = {}
for subtitle in (try_get(video, lambda x: x['subtitles']['urls'], list) or []): for subtitle in (try_get(video, lambda x: x['subtitles']['urls'], list) or []):
+11 -12
View File
@@ -40,14 +40,15 @@ class ARDMediathekBaseIE(InfoExtractor):
'This video is not available due to geoblocking', 'This video is not available due to geoblocking',
countries=self._GEO_COUNTRIES, metadata_available=True) countries=self._GEO_COUNTRIES, metadata_available=True)
self._sort_formats(formats)
subtitles = {} subtitles = {}
subtitle_url = media_info.get('_subtitleUrl') subtitle_url = media_info.get('_subtitleUrl')
if subtitle_url: if subtitle_url:
subtitles['de'] = [{ subtitles['de'] = [{
'ext': 'ttml', 'ext': 'ttml',
'url': subtitle_url, 'url': subtitle_url,
}, {
'ext': 'vtt',
'url': subtitle_url.replace('/ebutt/', '/webvtt/') + '.vtt',
}] }]
return { return {
@@ -262,7 +263,6 @@ class ARDMediathekIE(ARDMediathekBaseIE):
'format_id': fid, 'format_id': fid,
'url': furl, 'url': furl,
}) })
self._sort_formats(formats)
info = { info = {
'formats': formats, 'formats': formats,
} }
@@ -289,16 +289,16 @@ class ARDMediathekIE(ARDMediathekBaseIE):
class ARDIE(InfoExtractor): class ARDIE(InfoExtractor):
_VALID_URL = r'(?P<mainurl>https?://(?:www\.)?daserste\.de/(?:[^/?#&]+/)+(?P<id>[^/?#&]+))\.html' _VALID_URL = r'(?P<mainurl>https?://(?:www\.)?daserste\.de/(?:[^/?#&]+/)+(?P<id>[^/?#&]+))\.html'
_TESTS = [{ _TESTS = [{
# available till 7.01.2022 # available till 7.12.2023
'url': 'https://www.daserste.de/information/talk/maischberger/videos/maischberger-die-woche-video100.html', 'url': 'https://www.daserste.de/information/talk/maischberger/videos/maischberger-video-424.html',
'md5': '867d8aa39eeaf6d76407c5ad1bb0d4c1', 'md5': 'a438f671e87a7eba04000336a119ccc4',
'info_dict': { 'info_dict': {
'id': 'maischberger-die-woche-video100', 'id': 'maischberger-video-424',
'display_id': 'maischberger-die-woche-video100', 'display_id': 'maischberger-video-424',
'ext': 'mp4', 'ext': 'mp4',
'duration': 3687.0, 'duration': 4452.0,
'title': 'maischberger. die woche vom 7. Januar 2021', 'title': 'maischberger am 07.12.2022',
'upload_date': '20210107', 'upload_date': '20221207',
'thumbnail': r're:^https?://.*\.jpg$', 'thumbnail': r're:^https?://.*\.jpg$',
}, },
}, { }, {
@@ -371,7 +371,6 @@ class ARDIE(InfoExtractor):
continue continue
f['url'] = format_url f['url'] = format_url
formats.append(f) formats.append(f)
self._sort_formats(formats)
_SUB_FORMATS = ( _SUB_FORMATS = (
('./dataTimedText', 'ttml'), ('./dataTimedText', 'ttml'),
-1
View File
@@ -136,7 +136,6 @@ class ArkenaIE(InfoExtractor):
elif mime_type == 'application/vnd.ms-sstr+xml': elif mime_type == 'application/vnd.ms-sstr+xml':
formats.extend(self._extract_ism_formats( formats.extend(self._extract_ism_formats(
href, video_id, ism_id='mss', fatal=False)) href, video_id, ism_id='mss', fatal=False))
self._sort_formats(formats)
return { return {
'id': video_id, 'id': video_id,
-1
View File
@@ -73,7 +73,6 @@ class ArnesIE(InfoExtractor):
'width': int_or_none(media.get('width')), 'width': int_or_none(media.get('width')),
'height': int_or_none(media.get('height')), 'height': int_or_none(media.get('height')),
}) })
self._sort_formats(formats)
channel = video.get('channel') or {} channel = video.get('channel') or {}
channel_id = channel.get('url') channel_id = channel.get('url')
+20 -5
View File
@@ -65,6 +65,21 @@ class ArteTVIE(ArteTVBaseIE):
}, { }, {
'url': 'https://api.arte.tv/api/player/v2/config/de/LIVE', 'url': 'https://api.arte.tv/api/player/v2/config/de/LIVE',
'only_matching': True, 'only_matching': True,
}, {
'url': 'https://www.arte.tv/de/videos/110203-006-A/zaz/',
'info_dict': {
'id': '110203-006-A',
'chapters': 'count:16',
'description': 'md5:cf592f1df52fe52007e3f8eac813c084',
'alt_title': 'Zaz',
'title': 'Baloise Session 2022',
'timestamp': 1668445200,
'duration': 4054,
'thumbnail': 'https://api-cdn.arte.tv/img/v2/image/ubQjmVCGyRx3hmBuZEK9QZ/940x530',
'upload_date': '20221114',
'ext': 'mp4',
},
'expected_warnings': ['geo restricted']
}] }]
_GEO_BYPASS = True _GEO_BYPASS = True
@@ -180,13 +195,8 @@ class ArteTVIE(ArteTVBaseIE):
else: else:
self.report_warning(f'Skipping stream with unknown protocol {stream["protocol"]}') self.report_warning(f'Skipping stream with unknown protocol {stream["protocol"]}')
# TODO: chapters from stream['segments']?
# The JS also looks for chapters in config['data']['attributes']['chapters'],
# but I am yet to find a video having those
formats.extend(secondary_formats) formats.extend(secondary_formats)
self._remove_duplicate_formats(formats) self._remove_duplicate_formats(formats)
self._sort_formats(formats)
metadata = config['data']['attributes']['metadata'] metadata = config['data']['attributes']['metadata']
@@ -206,6 +216,11 @@ class ArteTVIE(ArteTVBaseIE):
{'url': image['url'], 'id': image.get('caption')} {'url': image['url'], 'id': image.get('caption')}
for image in metadata.get('images') or [] if url_or_none(image.get('url')) for image in metadata.get('images') or [] if url_or_none(image.get('url'))
], ],
# TODO: chapters may also be in stream['segments']?
'chapters': traverse_obj(config, ('data', 'attributes', 'chapters', 'elements', ..., {
'start_time': 'startTime',
'title': 'title',
})) or None,
} }

Some files were not shown because too many files have changed in this diff Show More