artem/yt-dlp
1
0
Fork 0
mirror of https://github.com/yt-dlp/yt-dlp.git synced 2026-09-04 10:11:38 +03:00
Code Issues Packages Projects Releases Wiki Activity
23,213 Commits 1 Branch 140 Tags
e25a148568b1259c3e718f07ecb88af31421a068
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
coletdjnz e25a148568 formatting 2025-05-18 09:58:32 +12:00
coletdjnz 93f8ffdad0 generalize provider setup 2025-05-17 13:44:49 +12:00
coletdjnz c1fd13a9b8 reorder extractor args 2025-05-17 13:39:42 +12:00
coletdjnzGitHubbashonly
 		37c5b6a6f2 Update README.md 
Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
 2025-05-17 13:38:15 +12:00
coletdjnz 2d594b4744 fixes from review 2025-05-17 13:37:16 +12:00
coletdjnz e7382323bc better cache key name for internal versioning 2025-05-17 13:37:16 +12:00
coletdjnzGitHubbashonly
 		ac58980b33 Update README.md 
Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
 2025-05-17 13:37:03 +12:00
coletdjnzGitHubbashonly
 		b4f0d272be Update yt_dlp/extractor/youtube/pot/README.md 
Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
 2025-05-17 13:20:19 +12:00
coletdjnz 26fbb09502 clarify class name requirements 2025-05-16 22:38:15 +12:00
coletdjnz 83edf67155 clean up docs 2025-05-16 21:01:07 +12:00
coletdjnz 2602a47685 Add never fetch pot policy 2025-05-16 20:54:54 +12:00
coletdjnz f7b821f490 Merge remote-tracking branch 'upstream/master' into feat/youtube/pot-provider-framework 2025-05-16 20:43:44 +12:00
coletdjnz 5506357315 improve docs 2025-05-16 20:42:57 +12:00
coletdjnz 50d12bbc5e cleanup some long lines 2025-05-16 20:30:06 +12:00
coletdjnz 6cdeec4332 flatten extractor arg docs 2025-05-16 20:16:02 +12:00
coletdjnz fcdf2b732f typing cleanup 2025-05-16 20:13:25 +12:00
coletdjnz 8045fed380 Revert back to pot_trace = true / false extractor arg 2025-05-16 19:17:23 +12:00
coletdjnzandGitHub 8cb08028f5 [ie/youtube] Detect and warn when account cookies are rotated (#13014) 
Related: https://github.com/yt-dlp/yt-dlp/issues/8227

Authored by: coletdjnz
 2025-04-27 12:16:34 +12:00
coletdjnzandGitHub 26feac3dd1 [ie/youtube] Add context to video request rate limit error (#12958) 
Related: https://github.com/yt-dlp/yt-dlp/issues/11426

Authored by: coletdjnz
 2025-04-25 16:11:07 +12:00
coletdjnz dd92c0df5b minor logging improvements 2025-04-20 20:21:21 +12:00
coletdjnz eb3680be9e typo lol 2025-04-19 11:14:51 +12:00
coletdjnz c1df426180 refactor: implement _SUPPORTED_EXTERNAL_REQEUST_FEATURES 2025-04-19 11:05:47 +12:00
coletdjnz 72a4a46152 refactor: provider _urlopen into _request_webpage, make pot_request optional 2025-04-18 17:21:26 +12:00
coletdjnz 7b0dd8b2d1 fix: expand webpo visitor id character set support 2025-04-18 16:51:05 +12:00
coletdjnz 4ee46531c9 Merge remote-tracking branch 'upstream/master' into feat/youtube/pot-provider-framework 2025-04-18 16:33:41 +12:00
coletdjnzandGitHub f484c51599 [ie/youtube] Add warning on video captcha challenge (#12939) 
Authored by: coletdjnz
 2025-04-18 11:40:39 +12:00
coletdjnzandGitHub 72ba487930 [ie/youtube:tab] Extract continuation from empty page (#12938) 
Fixes https://github.com/yt-dlp/yt-dlp/issues/12933 https://github.com/yt-dlp/yt-dlp/issues/8206

Authored by: coletdjnz
 2025-04-18 11:34:30 +12:00
coletdjnz d14c0fe223 [core] Add close hook API to YoutubeDL 2025-04-12 13:24:10 +12:00
coletdjnz 8e9a6553a6 [ie/youtube] Add log level for pot; do not call is_available() until necessary. 2025-04-12 11:34:31 +12:00
coletdjnz a0e036897b [ie/youtube] Move registry from globals to pot module 2025-04-12 09:28:22 +12:00
coletdjnz 71de622d19 [ie/youtube] Refactor project structure 2025-04-12 09:20:13 +12:00
coletdjnzandGitHub 0c857e5de8 Update yt_dlp/extractor/youtube/pot/README.md 2025-04-12 08:59:57 +12:00
coletdjnz 010e74410a [ie/youtube] Fix typos in README 2025-04-12 08:59:37 +12:00
coletdjnz 4e22fb6192 [ie/youtube] Rename cache register decorators 2025-04-12 08:58:56 +12:00
coletdjnz 8def006075 [ie/youtube] Rename get_setting to _configuration_arg 2025-04-12 08:53:29 +12:00
coletdjnz 18f89ca016 [ie/youtube] Remove max_size memory cache option 2025-04-12 08:46:23 +12:00
coletdjnz 038699cae6 [ie/youtube] Add a PO Token Provider Framework 2025-04-05 14:29:53 +13:00
coletdjnzandGitHub 6a6d97b2cb [ie/youtube:tab] Fix playlist continuation extraction (#12777) 
Fixes https://github.com/yt-dlp/yt-dlp/issues/12759

Authored by: coletdjnz
 2025-03-29 11:13:09 +13:00
coletdjnzandGitHub e67d786c7c [ie/youtube] Warn on DRM formats (#12593) 
Authored by: coletdjnz
 2025-03-16 10:28:16 +13:00
coletdjnzandGitHub 4432a9390c [ie/youtube] Split into package (#12557) 
Authored by: coletdjnz
 2025-03-13 17:37:33 +13:00
coletdjnzandGitHub 79ec2fdff7 [ie/youtube] Warn on missing formats due to SSAP (#12483) 
See https://github.com/yt-dlp/yt-dlp/issues/12482

Authored by: coletdjnz
 2025-02-28 19:33:31 +13:00
coletdjnzandGitHub 4445f37a7a [core] Load plugins on demand (#11305) 
- Adds `--no-plugin-dirs` to disable plugin loading
- `--plugin-dirs` now supports post-processors

Authored by: coletdjnz, Grub4K, pukkandan
 2025-02-23 11:00:46 +13:00
coletdjnzandGitHub d48e612609 [ie/youtube] Retry on more critical requests (#12339) 
Authored by: coletdjnz
 2025-02-19 00:39:51 +00:00
coletdjnzandGitHub 6b91d232e3 [ie/youtube] Use different PO token for GVS and Player (#12090) 
Authored by: coletdjnz
 2025-01-25 13:17:37 +13:00
coletdjnzandGitHub 326fb1ffaf [ie/youtube] Download tv client Innertube config (#12168) 
Authored by: coletdjnz
 2025-01-23 18:26:02 +13:00
coletdjnzandGitHub 75079f4e3f [ie/youtube] Refactor cookie auth (#11989) 
Authored by: coletdjnz
 2025-01-12 15:02:57 +13:00
coletdjnzandGitHub 712d2abb32 [ie/youtube] Use tv instead of mweb client by default (#12059) 
Authored by: coletdjnz
 2025-01-12 15:01:13 +13:00
coletdjnzandGitHub 0b6b7742c2 [ie/youtube] Fix DASH formats incorrectly skipped in some situations (#11910) 
Closes https://github.com/yt-dlp/yt-dlp/issues/11907
Authored by: coletdjnz
 2024-12-26 14:19:17 +13:00
coletdjnzandGitHub 9f42e68a74 [ie/youtube] Skip iOS formats that require PO Token (#11890) 
Partial fix for https://github.com/yt-dlp/yt-dlp/issues/11868

Authored by: coletdjnz
 2024-12-24 12:03:28 +13:00
coletdjnzandGitHub c7316373c0 [rh:websockets] Support websockets 14.0+ (#11616) 
Authored by: coletdjnz
 2024-11-24 10:30:00 +13:00
coletdjnzandGitHub b8635c1d47 [ie/youtube] Support logging in with OAuth (#11001) 
See: https://github.com/yt-dlp/yt-dlp/wiki/Extractors#logging-in-with-oauth

Authored by: coletdjnz
 2024-10-22 03:46:53 +00:00
coletdjnzandGitHub d1c4d88b2d [networking] Fix handler not being added to RequestError (#10955) 
Authored by: coletdjnz
 2024-09-08 19:32:44 +12:00
coletdjnzandGitHub 6f9e653743 [rh:websockets] Upgrade websockets to 13.0 (#10815) 
Fixes CI hanging

Authored by: coletdjnz
 2024-08-21 19:17:26 +12:00
coletdjnzandGitHub 150ecc45d9 [networking] Add legacy_ssl request extension (#10448) 
Supported by Urllib, Requests and Websockets request handlers. Ignored by CurlCFFI.

Also added couple cookie-related tests.

Authored by: coletdjnz
 2024-07-14 11:22:43 +12:00
coletdjnzandbashonly 42bfca00a6 [rh:curl_cffi] Support curl_cffi 0.7.X 
Authored by: coletdjnz
 2024-07-09 18:36:15 -05:00
coletdjnzandGitHub 96a134dea6 [ie/youtube] Extract upload timestamp if available (#9856) 
Closes #4962, Closes #9829
Authored by: coletdjnz
 2024-05-26 21:13:12 +00:00
coletdjnzandGitHub c36513f1be [rh:requests] Update to requests 2.32.0 (#9980) 
Authored by: coletdjnz
 2024-05-20 21:44:41 +00:00
coletdjnzandGitHub 53b4d44f55 [test] Fix connect timeout test (#9906) 
Fixes https://github.com/yt-dlp/yt-dlp/issues/9659

Authored by: coletdjnz
 2024-05-18 19:12:21 +12:00
coletdjnzGitHubbashonly
 		12d8ea8246 [ie/youtube] Remove android from default clients (#9553) 
Closes #9554
Authored by: coletdjnz, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
 2024-05-17 16:03:02 +00:00
coletdjnzandGitHub 3c7a287e28 [test] Add HTTP proxy tests (#9578) 
Also fixes HTTPS proxies for curl_cffi

Authored by: coletdjnz
 2024-05-11 10:06:58 +12:00
coletdjnzandGitHub e5d4f11104 [rh:websockets] Workaround race condition causing issues on PyPy (#9514) 
Authored by: coletdjnz
 2024-03-23 11:27:10 +13:00
coletdjnzbashonlySimon Sawickipukkandan
 		52f5be1f1e [rh:curlcffi] Add support for curl_cffi 
Authored by: coletdjnz, Grub4K, pukkandan, bashonly

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
Co-authored-by: bashonly <bashonly@protonmail.com>
 2024-03-16 23:15:11 -05:00
coletdjnzbashonlySimon Sawickipukkandan
 		0b81d4d252 Add new options --impersonate and --list-impersonate-targets 
Authored by: coletdjnz, Grub4K, pukkandan, bashonly

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
Co-authored-by: bashonly <bashonly@protonmail.com>
 2024-03-16 23:14:13 -05:00
coletdjnzandGitHub f849d77ab5 [test] Workaround websocket server hanging (#9467) 
Authored by: coletdjnz
 2024-03-16 16:57:21 +13:00
coletdjnzandGitHub 0085e2bab8 [rh] Remove additional logging handlers on close (#9032) 
Fixes https://github.com/yt-dlp/yt-dlp/issues/8922

Authored by: coletdjnz
 2024-02-18 11:32:34 +13:00
coletdjnzandGitHub 35f4f764a7 [rh:requests] Apply remove_dot_segments to absolute redirect locations 
Fixes https://github.com/yt-dlp/yt-dlp/issues/9020

Authored by: coletdjnz
 2024-01-21 10:03:33 +13:00
coletdjnzandGitHub 811d298b23 [networking] Remove _CompatHTTPError (#8871) 
Use `yt_dlp.networking.exceptions.HTTPError`.
`_CompatHTTPError` was to help with transition to the networking framework.

Authored by: coletdjnz
 2024-01-20 15:26:50 +13:00
coletdjnzandGitHub 69d3191495 [test] Skip source address tests if the address cannot be bound to (#8900) 
Fixes https://github.com/yt-dlp/yt-dlp/issues/8890

Authored by: coletdjnz
 2024-01-20 10:39:49 +13:00
coletdjnzandGitHub 196eb0fe77 [networking] Strip whitespace around header values (#8802) 
Fixes https://github.com/yt-dlp/yt-dlp/issues/8729
Authored by: coletdjnz
 2023-12-20 19:15:38 +13:00
coletdjnzandGitHub ccfd70f4c2 [rh:websockets] Migrate websockets to networking framework (#7720) 
* Adds a basic WebSocket framework
* Introduces new minimum `websockets` version of 12.0
* Deprecates `WebSocketsWrapper`

Fixes https://github.com/yt-dlp/yt-dlp/issues/8439

Authored by: coletdjnz
 2023-11-20 08:04:04 +00:00
coletdjnzandGitHub 8a8b54523a [rh:requests] Add handler for requests HTTP library (#3668) 
Adds support for HTTPS proxies and persistent connections (keep-alive)

Closes https://github.com/yt-dlp/yt-dlp/issues/1890
Resolves https://github.com/yt-dlp/yt-dlp/issues/4070
Resolves https://github.com/ytdl-org/youtube-dl/issues/32549
Resolves https://github.com/ytdl-org/youtube-dl/issues/14523
Resolves https://github.com/ytdl-org/youtube-dl/issues/13734

Authored by: coletdjnz, Grub4K, bashonly
 2023-10-13 23:33:00 +00:00
coletdjnzandGitHub eb5bdbfa70 [ie/youtube] Raise a warning for Incomplete Data instead of an error (#8238) 
Closes https://github.com/yt-dlp/yt-dlp/issues/8206

Adds `raise_incomplete_data` extractor arg to revert this behaviour and raise an error.

Authored by: coletdjnz
Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
 2023-10-03 06:42:30 +00:00
coletdjnzandGitHub 9d6254069c Update to ytdl-commit-66ab08 (#8128) 
[utils] Revert bbd3e7e, updating docstring, test instead
 https://github.com/ytdl-org/youtube-dl/commit/66ab0814c4baa2dc79c2dd5287bc0ad61a37c5b9

Authored by: coletdjnz
 2023-09-20 19:14:10 +00:00
coletdjnzandGitHub 20fbbd9249 [networking] Fix various socks proxy bugs (#8065) 
- Fixed support for IPv6 socks proxies
- Fixed support for IPv6 over socks5
- Fixed --source-address not being obeyed for socks4 and socks5
- Fixed socks4a when the destination address is an IPv4 address

Closes https://github.com/yt-dlp/yt-dlp/issues/7959
Fixes https://github.com/ytdl-org/youtube-dl/issues/15368

Authored by: coletdjnz
Co-authored-by: Simon Sawicki <accounts@grub4k.xyz>
Co-authored-by: bashonly <bashonly@bashonly.com>
 2023-09-18 07:33:26 +00:00
coletdjnzandGitHub fcd6a76adc [tests] Add tests for socks proxies (#7908) 
Authored by: coletdjnz
 2023-08-25 07:10:44 +00:00
coletdjnzandGitHub 378ae9f9fb [ie/youtube] Fix consent cookie (#7774) 
Fixes #7594

Authored by: coletdjnz
 2023-08-12 04:26:08 +00:00
coletdjnzandGitHub db7b054a61 [networking] Add request handler preference framework (#7603) 
Preference functions that take a request and a request handler instance can be registered to prioritize different request handlers per request.

Authored by: coletdjnz
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
 2023-08-04 22:17:48 +00:00
coletdjnz 546b2c28a1 [ie/youtube] Fix player_params arg being converted to lowercase 
Fix bug in ba06d77a31

Authored by: coletdjnz
 2023-07-30 10:50:25 +12:00
coletdjnzandGitHub ba06d77a31 [ie/youtube] Add player_params extractor arg (#7719) 
Authored by: coletdjnz
 2023-07-29 06:20:42 +00:00
coletdjnzandGitHub 4bf912282a [networking] Remove dot segments during URL normalization (#7662) 
This implements RFC3986 5.2.4 remove_dot_segments during the URL normalization process.

Closes #3355, #6526

Authored by: coletdjnz
 2023-07-28 22:40:20 +00:00
coletdjnzandGitHub bbeacff7fc [networking] Ignore invalid proxies in env (#7704) 
Authored by: coletdjnz
 2023-07-27 20:26:02 +05:30
coletdjnzandGitHub 95abea9a03 [test] Fix httplib_validation_errors test for old Python versions (#7677) 
Fixes https://github.com/yt-dlp/yt-dlp/issues/7674

Authored by: coletdjnz
 2023-07-24 19:18:52 +00:00
coletdjnzandGitHub 86aea0d3a2 [networking] Add strict Request extension checking (#7604) 
Authored by: coletdjnz
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
 2023-07-23 05:17:15 +00:00
coletdjnzandpukkandan 3d2623a898 [compat, networking] Deprecate old functions (#2861) 
Authored by: coletdjnz, pukkandan
 2023-07-15 16:18:35 +05:30
coletdjnzandpukkandan 227bf1a33b [networking] Rewrite architecture (#2861) 
New networking interface consists of a `RequestDirector` that directs
each `Request` to appropriate `RequestHandler` and returns the
`Response` or raises `RequestError`. The handlers define adapters to
transform its internal Request/Response/Errors to our interfaces.

User-facing changes:
- Fix issues with per request proxies on redirects for urllib
- Support for `ALL_PROXY` environment variable for proxy setting
- Support for `socks5h` proxy
   - Closes https://github.com/yt-dlp/yt-dlp/issues/6325, https://github.com/ytdl-org/youtube-dl/issues/22618, https://github.com/ytdl-org/youtube-dl/pull/28093
- Raise error when using `https` proxy instead of silently converting it to `http`

Authored by: coletdjnz
 2023-07-15 16:18:35 +05:30
coletdjnzandGitHub 1ba6fe9db5 [ie/youtube:tab] Detect looping feeds (#6621) 
Closes https://github.com/yt-dlp/yt-dlp/issues/5555

Note: the first page may still be repeated, however this is better than nothing.

Authored by: coletdjnz
 2023-07-15 03:20:24 +00:00
coletdjnzandpukkandan f8b4bcc0a7 [core] Prevent Cookie leaks on HTTP redirect 
Ref: https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-v8mc-9377-rwjj

Authored by: coletdjnz
 2023-07-06 23:14:39 +05:30
coletdjnzandGitHub 90db9a3c00 [extractor/youtube:stories] Remove (#7459) 
YouTube killed them

https://web.archive.org/web/20230630153050/https://support.google.com/youtube/thread/217640760
 2023-07-06 19:02:41 +05:30
coletdjnzandGitHub 4dc4d8473c [extractor/youtube] Ignore incomplete data for comment threads by default (#7475) 
For both `--ignore-errors` and `--ignore-errors only_download`. Pass `--no-ignore-errors` to not ignore.

Closes https://github.com/yt-dlp/yt-dlp/issues/7474

Authored by: coletdjnz
 2023-07-03 10:47:10 +00:00
coletdjnzandGitHub fcbc9ed760 [extractor/youtube:tab] Support shorts-only playlists (#7425) 
Fixes https://github.com/yt-dlp/yt-dlp/issues/7424

Authored by: coletdjnz
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
 2023-06-29 23:26:27 +00:00
coletdjnzandpukkandan 8213ce28a4 [extractor/youtube] Extract channel_is_verified (#7213) 
Authored by: coletdjnz
 2023-06-08 19:15:39 +05:30
coletdjnzandGitHub 2fb35f6004 [extractor/youtube] Support shorter relative time format (#7191) 
See: https://github.com/TeamNewPipe/NewPipeExtractor/issues/1067

Authored by: coletdjnz
 2023-06-03 06:33:51 +00:00
coletdjnzandGitHub c35448b7b1 [extractor/youtube] Extract more metadata for comments (#7179) 
Adds new comment fields:
* `author_url` - The url to the comment author's page
* `author_is_verified` - Whether the author is verified on the platform
* `is_pinned` - Whether the comment is pinned to the top of the comments

Closes https://github.com/yt-dlp/yt-dlp/issues/5411

Authored by: coletdjnz
 2023-06-01 08:43:32 +00:00
coletdjnzandGitHub 18f8fba7c8 [extractor/youtube] Fix continuation loop with no comments (#7148) 
Deep check the response for incomplete data.

Authored by: coletdjnz
 2023-05-31 07:08:28 +00:00
coletdjnz 93e12ed76e [extractor/youtube] Extract uploader metadata for feed/playlist items 
Fixes https://github.com/yt-dlp/yt-dlp/issues/7104

Authored by: coletdjnz
 2023-05-28 11:31:45 +12:00
coletdjnzandGitHub daafbf49b3 [core] Support decoding multiple content encodings (#7142) 
Authored by: coletdjnz
 2023-05-27 10:40:05 +00:00
coletdjnzandGitHub 3f66b6fe50 [core] Workaround erroneous urllib Windows proxy parsing (#7092) 
Convert proxies extracted from windows registry to http for older Python versions.
See: https://github.com/python/cpython/issues/86793

Authored by: coletdjnz
 2023-05-27 07:17:27 +00:00
coletdjnzandGitHub b87e01c123 [cookies] Move YoutubeDLCookieJar to cookies module (#7091) 
Authored by: coletdjnz
 2023-05-27 07:08:19 +00:00
coletdjnzandGitHub 08916a49c7 [core] Improve HTTP redirect handling (#7094) 
Aligns HTTP redirect handling with what browsers commonly do and RFC standards. 

Fixes issues https://github.com/yt-dlp/yt-dlp/commit/afac4caa7db30804bebac33e53c3cb0237958224 missed.

Authored by: coletdjnz
 2023-05-27 07:06:13 +00:00
coletdjnzandGitHub 955c89584b [core] Deprecate internal Youtubedl-no-compression header (#6876) 
Authored by: coletdjnz
 2023-05-20 22:55:09 +00:00