github.com/ArchiveTeam/ludios_wpull commits | Ecosyste.ms: OpenCollective

abstract.client: Split _acquire_connection for Request or host & port args

75f8efc940b0ba9e765b67063c48eab55b8343a9 authored almost 9 years ago by Christopher Foo <[email protected]>

http.web: Split main user function into two to match Client. Add typing.

5906e1ce670fa035e98cd2898653471fb8a37107 authored almost 9 years ago by Christopher Foo <[email protected]>

http.client: Rename the main functions. Use enum to track session state.

Add add typing.

d3bc8cf32408582e71f735d3a5ce99235a289029 authored almost 9 years ago by Christopher Foo <[email protected]>

abstract.client: Handle connection pool concerns in BaseSession

Add typing.

38b217ac092300df15f97b3b8ff8600b133d31b6 authored almost 9 years ago by Christopher Foo <[email protected]>

ftp.ls.date: Add function typing

882a9335bb7a9b18a02e9b4a1d277138e8512832 authored almost 9 years ago by Christopher Foo <[email protected]>

Fix up connection & pool tests

cebe4c52aa46f04e4daa1d06eeb363e16330e35c authored almost 9 years ago by Christopher Foo <[email protected]>

pool: Fix updated Resolver references. Add function typing annotations.

adacd640bc25756ed2cd1304bf1eb6a62877135c authored almost 9 years ago by Christopher Foo <[email protected]>

connection: Add function typing annotations

1e58f56b8d0abe9dd92a107555128ca038aedb59 authored almost 9 years ago by Christopher Foo <[email protected]>

connection: Remove use match_hostname alias. Handle CERT_OPTIONAL case.

005ff55b61de81bee30976767727900f954f38c9 authored almost 9 years ago by Christopher Foo <[email protected]>

network: Move connection pool classes to pool module

8041c6bcb73d40cb96faf5e4a1be3f91048bb005 authored almost 9 years ago by Christopher Foo <[email protected]>

Move bandwidth, connection, dns modules into network package

f99058e95feca6a7f4a24b09506e16a44c2f387b authored almost 9 years ago by Christopher Foo <[email protected]>

travis config: Blacklist topic/modernize_2016

a41410c87bfab58ae974ecb49c36ca37ed91b3cd authored almost 9 years ago by Christopher Foo <[email protected]>

Sort requirements and add "typing" dependency

0955c1592d2ced0430650a216fe60d640dda1507 authored almost 9 years ago by Christopher Foo <[email protected]>

dns: Rewrite Resolver to return structured data and DNS RR text format.

df513bf803d81cd161393c37a093b8fcf5ea0fbc authored almost 9 years ago by Christopher Foo <[email protected]>

url: Normalize IP addresses

96ab757b65462fce8f9325b5c3a8cd25a87cffc5 authored almost 9 years ago by Christopher Foo <[email protected]>

changelog: Note Lua and Python 3.2/3.3 removal

[ci skip]

c8a8fe63a00fdb9388e0a34e06657a51991b89d9 authored almost 9 years ago by Christopher Foo <[email protected]>

Merge branch 'develop' into topic/modernize_2016

65f6fdd49c883d5ec0038d82549c510bbe7bb232 authored almost 9 years ago by Christopher Foo <[email protected]>

Bump version 1.2.3

60842fc656d85729ff91722f49e4b10f375090bd authored almost 9 years ago by Christopher Foo <[email protected]>

Merge branch 'develop'

09c519507e40eead12d1fcbe957f9a5cb40ca6ec authored almost 9 years ago by Christopher Foo <[email protected]>

changelog: Update latest to version 1.2.3

[ci skip]

77b24f70296f24b7541b7de765a921f896cdfa60 authored almost 9 years ago by Christopher Foo <[email protected]>

app_test: Skip PhantomJS tests under Travis

070cfd9f467f85ee97f6f90ac03044024531e6df authored almost 9 years ago by Christopher Foo <[email protected]>

document tests: Skip testing UTF 16/32 under Lxml for now

b44d454f3a6cfde503c14af7e926abb271f7b224 authored almost 9 years ago by Christopher Foo <[email protected]>

scraper tests: Update for urljoin change in Py 3.5

de7c183e364eef0a44179c14845705c6a4328904 authored almost 9 years ago by Christopher Foo <[email protected]>

testing.goodapp: Use int() on HTTP constant for Py 3.5

7171732f116964f8d2638507fe88e214a9c33537 authored almost 9 years ago by Christopher Foo <[email protected]>

doc: Remove references to Lua. Update Py versions. Regenerate API docs.

f2143864488f57dd0ee05c7c0ce0f27b245bd56b authored almost 9 years ago by Christopher Foo <[email protected]>

Remove Trollius dependency. Upgrade syntax to "yield from"

58e3cfdbb9e4594cbd6b6381fa5cddf9e93f5113 authored almost 9 years ago by Christopher Foo <[email protected]>

Remove cx_freeze build support

c65f36bb705d5b3e9ef403daa987b706d418b935 authored almost 9 years ago by Christopher Foo <[email protected]>

Remove backport.textwrap

5201ca8ae2172c3597acab6bf6432c475659b27b authored almost 9 years ago by Christopher Foo <[email protected]>

ftp.ls: Make ListingParser subclass of ListingParser

6ccb36ffce1c7c7de1b345b2c3b7303c20f885fb authored almost 9 years ago by Christopher Foo <[email protected]>

ftp listing: Return as listing results as generator

So it may be used for long streaming listings

e5806c345f072373e501d6ae90ed795bf71f763f authored almost 9 years ago by Christopher Foo <[email protected]>

Move 'http' and 'ftp' packages under 'protocol' package.

015cadf07bf44b0c5b4c2baa47db2fa4b63caf44 authored almost 9 years ago by Christopher Foo <[email protected]>

Remove PyPy Py 3.2 & 3.3 from Travis config and setup.py

9bcaed0e06f71b16e38a612ba9a0028a3add7c59 authored almost 9 years ago by Christopher Foo <[email protected]>

Remove Lua support

d7e1f5580f424729d5234a58cc396e8dff9920d2 authored almost 9 years ago by Christopher Foo <[email protected]>

url: Check for control chars instead of using isprintable()

Closes: chfoo/wpull#298

c7faa691985950db9ed2b81d7758ad943e2f0970 authored almost 9 years ago by Christopher Foo <[email protected]>

ftp: Check and correct when test date is in the future in asserts

d28cc8fed89aea63615816c2af8774db0ee11176 authored almost 9 years ago by Christopher Foo <[email protected]>

changelog: Add entry for FTP login fix

8f719e14fe5a4a935b414eceeb7a103deffed756 authored almost 9 years ago by Christopher Foo <[email protected]>

ftp: Don't check for password error if username is only required.

Closes chfoo/wpull#292

c3c9efe0e056a6fa2a84bacb99ad4c491785eac8 authored almost 9 years ago by Christopher Foo <[email protected]>

changelog: Add pending Lua & Python 3.2/3.3 deprecation

c5c567335fe095e9fe7d0125023ddce40702ccb8 authored almost 9 years ago by Christopher Foo <[email protected]>

Bump version 1.2.2

36f83c308a5a07d989c1b28871ec4aea8ba2bb1a authored about 9 years ago by Christopher Foo <[email protected]>

Merge branch 'develop'

176a3d9918728238b8a44b2ac48255d166d7eef5 authored about 9 years ago by Christopher Foo <[email protected]>

changelog: Update latest to 1.2.2

[ci skip]

8ca1aa7b9a2df53538bc28df41a0445fca8185cd authored about 9 years ago by Christopher Foo <[email protected]>

string: Handle ValueError codec error on Py 3.5

ccef9ecf55a5cfd1fd989ec963fca9a36c616719 authored about 9 years ago by Christopher Foo <[email protected]>

travis: Fix "and" condition and Py 3.5

e7d8f96dfc1247b341d943360dbf186e80fe2c58 authored about 9 years ago by Christopher Foo <[email protected]>

engine: Use Queue if JoinableQueue has been removed

Closes chfoo/wpull#289

3e97037ec6ee375e2bf964c0e59dd34f9239de9b authored about 9 years ago by Christopher Foo <[email protected]>

Merge branch 'ludios-fix-no-sslv3' into develop

089e931f6fbdba1e90cf0e16e808ec4b64de4940 authored about 9 years ago by Christopher Foo <[email protected]>

travis: Disable running coverage under Py 3.2

coverage 4.0+ does not support 3.2

b9753262c6ac587ef49cbff139ed1564be2133b8 authored about 9 years ago by Christopher Foo <[email protected]>

options: Don't create tuples for SSL modes

7a2ad14ef191ba67015741b587c39f4aaa999108 authored about 9 years ago by Ivan Kozik <[email protected]>

recorder: Allow ipv4-mapped addresses in assert

Closes chfoo/wpull#286

36fdf27e50a59fd3f8c6110de69825b19395cd1f authored about 9 years ago by Christopher Foo <[email protected]>

setup.py: Remove explicit psutil requirement

psutil may not always be available and we check for this at import time
already

Closes chfoo/wp...

1546fe50079620e1a9866e54c88adb566a20bf7f authored about 9 years ago by Christopher Foo <[email protected]>

url: Validate port number and Raise ValueError if invalid

Closes chfoo/wpull#278

91e0a1d01cdf312255da6fe83d405559a45c3d92 authored about 9 years ago by Christopher Foo <[email protected]>

recorder: Write output document only after response is done.

Closes chfoo/wpull#275

f7eb2c0528eab3edc51ee68e7f827ea41a5f26c2 authored about 9 years ago by Christopher Foo <[email protected]>

options: Don't crash when OpenSSL lacks SSLv3 support

540f3355405ce8c215d43e6c4f25c19f4e4d0e56 authored about 9 years ago by Ivan Kozik <[email protected]>

setup.py: Add PyPI FTP classifier

[ci skip]

ee5c4650ef676e5382192910999cfbaa2cb9e06b authored over 9 years ago by Christopher Foo <[email protected]>

Bump version 1.2.1

9df49d878129757b00e3289ca948202ee2c25c74 authored over 9 years ago by Christopher Foo <[email protected]>

Merge branch 'develop'

baaeb6216dc5d0060a96c7eceae96ae5cf831310 authored over 9 years ago by Christopher Foo <[email protected]>

changelog: Update latest to 1.2.1

[ci skip]

91cc0288aa923e7b537a32107733ced82b1088f8 authored over 9 years ago by Christopher Foo <[email protected]>

builder: Check and use as-is stdin input file

Closes chfoo/wpull#272

bdfd8d9d3813ec29b3b103f207506fa8c16781ad authored over 9 years ago by Christopher Foo <[email protected]>

changelog: Add youtube-dl ipv4/certs entry

[ci skip]

ea6267f1a30ace5573116105874fc607b872ec40 authored over 9 years ago by Christopher Foo <[email protected]>

Merge branch 'yipdw-topic/youtube-dl-ipv4-no-tls-flags' into develop

Conflicts:
wpull/version.py

84c0fc00fe2ae097ad739959f2aa597fbea77bce authored over 9 years ago by Christopher Foo <[email protected]>

dns: Handle OverflowError with large ports

Closes chfoo/wpull#270

64e21cf4bdaac6b6c7da832406dbe6660f6d0480 authored over 9 years ago by Christopher Foo <[email protected]>

youtube-dl: Infer --force-ipv4 and --no-check-certificate flags.

If wpull's in IPv4-only mode, youtube-dl should also run in that mode
to maximize consistent beh...

9a239c31b82f8b90c2df2ecc63e46ab7eaa3e0a1 authored over 9 years ago by David Yip <[email protected]>

Bump version 1.2

34c42cd95547fc644f89bb810c21e16764dadea5 authored over 9 years ago by Christopher Foo <[email protected]>

Merge commit 'b8e6b8d60039da4ad8fe5b86f24ad792e2f65f26'

5ec391645a10d0345e8414573cbdd4b8c615d97c authored over 9 years ago by Christopher Foo <[email protected]>

Bump version 1.3a1

[ci skip]

416073b82d9d042360e468aa93576784cac4cf94 authored over 9 years ago by Christopher Foo <[email protected]>

changelog: Update latest to 1.2

[ci skip]

b8e6b8d60039da4ad8fe5b86f24ad792e2f65f26 authored over 9 years ago by Christopher Foo <[email protected]>

Reapply ".travis.yml: Switch to container infrastructure."

Package name should be fixed now.

87f6d3d054a1b071759e928b3f144592f304e6e7 authored over 9 years ago by Christopher Foo <[email protected]>

Merge branch 'issue/222-ftp_glob_2' into develop

Closes chfoo/wpull#222

364d3d8ae66e817a2244dd45cedbf3b86444e7a0 authored over 9 years ago by Christopher Foo <[email protected]>

Implement FTP filename globbing

5ad10823bdb58d1174390cf4b9606342d30e3fc2 authored over 9 years ago by Christopher Foo <[email protected]>

item: Add level arg to add_child_urls() & child_url_record()

ada2020d528382491e0cd781707aad67cb939717 authored over 9 years ago by Christopher Foo <[email protected]>

url: Add URLInfo.split_path()

c95b608150d01195adb4c6f9afd804b9ed530c38 authored over 9 years ago by Christopher Foo <[email protected]>

builder: Add argv to warcinfo extra fields

Closes chfoo/wpull#267

b44d5a3d3f23399d26dc63bba303811f1f32b68d authored over 9 years ago by Christopher Foo <[email protected]>

recorder.warc: Wrap warcinfo fields to 1024 chars

2252d30a8b9214fc79e12f2d18cf18036b2fde66 authored over 9 years ago by Christopher Foo <[email protected]>

namevalue.NameValueRecord: Add wrap_width arg

6087bfa92c9024cad5e4b7f63e5478c0d70acd43 authored over 9 years ago by Christopher Foo <[email protected]>

Hook up --keep-session-cookies instead of always True.

Closes chfoo/wpull#266

7352c90c0b870a98cdbe26e3cb4ab4c7ce97d73c authored over 9 years ago by Christopher Foo <[email protected]>

cookie: Add BetterMozillaCookieJar copied from Lib w/ session cookie support

Add magic_re from and remove RelaxedMozillaCookieJar

Closes chfoo/wpull#261

14af51651f3c94782d8eb57b1d48158ab4d88cfc authored over 9 years ago by Christopher Foo <[email protected]>

resmon: Use friendlier ImportError warning.

9eca942e67286b4701bd858ad2876b84af1da253 authored over 9 years ago by Christopher Foo <[email protected]>

test: Run fuzzers with faulthandler enabled

[ci skip]

800415281cb7da32345a55874b380aad57791644 authored over 9 years ago by Christopher Foo <[email protected]>

ftp: Use UTF8 w/ surrogateescape instead of Latin1

Closes chfoo/wpull#264

bbb6b8de5b636c816de2a5722932fe633dfe66c3 authored over 9 years ago by Christopher Foo <[email protected]>

path: Don't percent-encode bytes 128-159 if not ASCII

d82c6a07cd21ad41b6288d27dffdd2d210b11855 authored over 9 years ago by Christopher Foo <[email protected]>

builder: Use PickleStream to keep sorted URLInfos.

Closes chfoo/wpull#262

41571d59ed343603565909e8e5892520312f166f authored over 9 years ago by Christopher Foo <[email protected]>

util: Add PickleStream

ecd0a90062cf132c2a30277921b54170b250ea18 authored over 9 years ago by Christopher Foo <[email protected]>

changelog: Add sqlalchemy 1.0 and pypy fix note.

[ci skip]

27d212606a94a1d56ffe9e02475931171ee7813a authored over 9 years ago by Christopher Foo <[email protected]>

Revert "Reapply ".travis.yml: Switch to container infrastructure.""

This reverts commit c371390fe00fffcc90fa6a1ccfd3fefa17b4ab4b.

Not fixed yet.

727dc7f33b89f74961d1cd961d188de432e93db9 authored over 9 years ago by Christopher Foo <[email protected]>

Reapply ".travis.yml: Switch to container infrastructure."

Package name should be fixed now.

c371390fe00fffcc90fa6a1ccfd3fefa17b4ab4b authored over 9 years ago by Christopher Foo <[email protected]>

databases.sqltable: Fix update query for sqlalchemy 1.0 and pypy.

34799700e31ca1508d3216903739a1421099c10b authored over 9 years ago by Christopher Foo <[email protected]>

Revert ".travis.yml: Switch to container infrastructure."

This reverts commit 6bf0a66c4dfdb771bde25a58ac12bdecc5a7b9ef.

liblua isn't whitelisted correctl...

0fe0c31b2d06cbeb60268f5c55abe7f3d1eae2ba authored over 9 years ago by Christopher Foo <[email protected]>

.travis.yml: Fix liblua package name

9772950fc7d4edac1d6c09e8a65b092ba78be8f9 authored over 9 years ago by Christopher Foo <[email protected]>

.travis.yml: Switch to container infrastructure.

6bf0a66c4dfdb771bde25a58ac12bdecc5a7b9ef authored over 9 years ago by Christopher Foo <[email protected]>

Add default "none" as choice to --prefer-family.

Closes chfoo/wpull#154

a0f29cb7cd75972a8d5e10ace169d7080deb734f authored over 9 years ago by Christopher Foo <[email protected]>

Merge branch 'issue/154-happy_eyeballs_2' into develop

878ad74e77e0089d42928209b447ce5bfb35d5d2 authored over 9 years ago by Christopher Foo <[email protected]>

connection_test: Add test_happy_eyeballs_table

5c614c83c390ed3af7ba30effc27e1bae692a833 authored over 9 years ago by Christopher Foo <[email protected]>

ftp,http: Reconnect before passing address to recorders

ftp.stream: Change reconnect() into read_welcome_message()
app_test: Add additional mock methods...

a001c12da472452a2ebddb85e9b7fa3e16b3686c authored over 9 years ago by Christopher Foo <[email protected]>

connection: Add HappyEyeballsConnection & HappyEyeballsTable

re chfoo/wpull#154

5fcc9938638c6f319e55e6a00502b647e08c4e7e authored over 9 years ago by Christopher Foo <[email protected]>

Bump version 1.1.1

973ddef5109c57aeca4c882450d96ce9ad1e511b authored over 9 years ago by Christopher Foo <[email protected]>

Merge branch 'develop'

Conflicts:
wpull/version.py

b385d696a6b322d90232592ad5d01741cea0d58c authored over 9 years ago by Christopher Foo <[email protected]>

changelog: Update latest to 1.1.1

[ci skip]

e41336326657e859e18d47ae11c674dcee97eb79 authored over 9 years ago by Christopher Foo <[email protected]>

recorder.warc: Assert request address is an IP address

43d98a3f8b3f787423dbff4b1c4d8587ff6bb851 authored over 9 years ago by Christopher Foo <[email protected]>

Change log/progresss message "none" to "unspecified" to match Wget

9fcf5011a110627523734a0868f57e32fabf81aa authored over 9 years ago by Christopher Foo <[email protected]>

connection: Refactor Connection to BaseConnection

e774e1b04f66d187ddcad2856cb4e15fdf81bad9 authored over 9 years ago by Christopher Foo <[email protected]>

dns: Remove global cache and use instance cache.

Use a per instance cache to avoid potential side effects that may
happen, for example, during un...

b90d044faa7175e3fbca543c046e8bfc5f32b66d authored over 9 years ago by Christopher Foo <[email protected]>