github.com/ArchiveTeam/ludios_wpull commits | Ecosyste.ms: OpenCollective

builder.py: Put back ignore-length and no-http-keep-alive opts.

114932a55d494290d67845f13b2267c280c9ce71 authored over 10 years ago

app_test.py: Fix up erroring code.

24c271b3333808eb314a86e378d6311526a75c3c authored over 10 years ago

engine.py: Count items out using the queue.

f18283f62047ecd54aadfdaafb75f46058c6fe12 authored over 10 years ago

connection.py: Catch SSL verfication error by checking error msg str.

3ac9718ff3dcc58ef0e12e68b1d417414ddba2bc authored over 10 years ago

Continously force feed phantomjs stdin data to not block.

bf76b469d64ca56f132f1056d94862a505c9fabe authored over 10 years ago

Fix up attributes and calls. Add back in robots.txt handling.

11519fc0f9d0cd760ed8a67193760fc72d4dbf31 authored over 10 years ago

Fix up proxy and unit tests.

92e427a2484ab87ba4172484ee8a7c1717555bb6 authored over 10 years ago

Remove obsolete actor, iostream, async modules.

186ca925d92bc5a807187958b9083c9c1d6a772b authored over 10 years ago

phantomjs: Use stdin/stdout for communication instead of WebSocket.

38f435f2858d0449b163b400979965a60ef748cc authored over 10 years ago

Misc attribute fix ups.

6075685aacd891e21e484df907030c32e5295bed authored over 10 years ago

document_test.py: Fix up Response args

21416c6079910b28b87974778f409fc30662efb1 authored over 10 years ago

Merge branch 'develop' into topic/asyncio

Conflicts:
wpull/http/client.py
wpull/http/connection.py
wpull/phantomjs.py

4ccc25ab369fabcc0062fe6efc4a890f7ce447aa authored over 10 years ago

Rewrite proxy.py. http: Allow raw read of chunked content.

fc0b4056daaeb98aa788dc127f0c81338c55f4c6 authored over 10 years ago

Fix up attributes.

14bb875a4799c03ebbb369f723d2e3dc331c2c2b authored over 10 years ago

WIP fix up WebProcessor

50b9d105fbe0b60e200872b12c8f54e567365b63 authored over 10 years ago

Split processor module into package.

958828196a61a058d3f2bd2025ce87121270f55b authored over 10 years ago

Fix up engine.

f23917bdaf21ee7306dc858e08f7bba40a2af688 authored over 10 years ago

WIP Fix up builder.

c61e6103ad8c9c2458c0a624909d01eced02e979 authored over 10 years ago

travis.yml: Blacklist topic/asyncio branch.

So don't have to write ci skip so many times

6d2fbf020690fabc1063035ebb89ae82afa79c54 authored over 10 years ago

Fix up recorder_test.py and related

[ci skip]

64dda8a156fd4a4c5877fbd8a99290fb9d3b9284 authored over 10 years ago

Fix up wrapper_test

[ci skip]

ab1f8c09f123d3bb1d679931ee05f51b17af9926 authored over 10 years ago

Fix up scraper

[ci skip]

6eefc8b4b58d5b62b29a6668951f719309b99516 authored over 10 years ago

http.request: Split Request into Request and RawRequest

[ci skip]

0eceb812c5ae267eeb22e32f992d5feb61c58739 authored over 10 years ago

Bump version to 0.36.4

b20cbc95e77aed1494906ad0b5dc668f7f48ebb4 authored over 10 years ago

Merge branch 'develop'

af7e2b3deb2a03cab38973749efdd3d4f524baa6 authored over 10 years ago

changelog: Update latest to 0.36.4

[ci skip]

90201850984ab166d4ea4b21c10aa835f9bfc458 authored over 10 years ago

changelog: Add entry: Tornado 4.0 support.

Closes chfoo/wpull#142

189943e469604575cb31803affa19c1e6b65640f authored over 10 years ago

Merge branch 'issue/142-tornado4' into develop

2fffc154316e2b4dc1b13bafd0a63c87ef3995db authored over 10 years ago

backport.conf: Add websocket.py to nofix=str

7494f6d948628157eb921b331e2ce454b9a66c46 authored over 10 years ago

app_test.py: Use coroutine version of fetch

Avoid tornado getting confused and stopping/starting the ioloop
incorrectly

017db5932d8285ea4e5aa4a0fe5f2923f43b179c authored over 10 years ago

phantomjs: Don't call set_exit_callback

Avoid signal/ioloop/atexit/AttributeError problems.

b7dae043d2ae338122d535f0b113c88944bff7fc authored over 10 years ago

requirements.txt: Set tornado>=3.1.1,<5.0

6d1be19c7b9b47ae7bc7a2c540844406a64da708 authored over 10 years ago

backport.conf: Update for .thirdparty.tornado

1d22ed7b7499bd579506ac1f2a65315e8e4cb4b0 authored over 10 years ago

phnatomjs: Copy and use websocket.py

re: chfoo/wpull#142

websocket.py is from
tornadoweb/tornado@1c36307463b1e8affae100bf9386948e6c1...

65c04ff679de0150f06aeddb958107e325dc7829 authored over 10 years ago

http.connection: Don't early log all exceptions.

Let handlers log exceptions at the expense of missing original traceback
info.

0e1f46944fc88f5ef407af5ffa9edc767e4c4cb0 authored over 10 years ago

async: Ensure future is consumed after timeout.

455a28b0f090acc6604536164a884839a13b438e authored over 10 years ago

.http.connection: Don't leak tracebacks across exception contexts

Python 3 seems to want to store the traceback on the exception object.

Side effect is that exce...

243ba0d52b98e7df79246f77f5136f1abdc1da10 authored over 10 years ago

decompression: Check for gzip using only first byte

Closes chfoo/wpull#155

4e2d5fb8885b3c36106d6ee7d712d241a1a26d2c authored over 10 years ago

decompression: Copy and mod tornado.util.GzipDecompressor

[ci skip]

8de02bd469e2d27e50473f61649cb3769a96b726 authored over 10 years ago

http.connection: Don't use bare except.

b24510f729be1a859c2a7e00a9a793b1a7630a4a authored over 10 years ago

dns.py: Add missing .async import.

22de80e89fd4d18b58c02e09c33c54fc3a71c3d5 authored over 10 years ago

WIP Rewrite RobotsTxtRichClientSession into new .http.robots module.

[ci skip]

bde05c02d6bb64d5e19a60d7e17fff3680c57eec authored over 10 years ago

WIP http.web asyncio support

[ci skip]

ca78db76248bbe2402e13c317fbb147807b029ac authored over 10 years ago

testing: Support goodapp with asyncio/trollius

[ci skip]

28b93872ed0e5bc365bc4b0ef1c8c6b25027b19e authored over 10 years ago

http.request: Add copy method

[ci skip]

912b15ce30af0ce63413ba600a7d9d6af4967a87 authored over 10 years ago

http.redirect: Fix up response attribute access

[ci skip]

0f15c325f7f0ee75a4ef3b9d7f2be50c5438a8aa authored over 10 years ago

cookie: Discard non-ASCII cookie values

Closes chfoo/wpull#145

85d7af952bc379b54d24e89c73a6a1941f50e1c5 authored over 10 years ago

cookie: Rename CookieLimitsPolicy to DeFactoCookiePolicy

25fa23663c919843980d01b41e0963675f03d2ff authored over 10 years ago

Bump version to 0.36.3

0c79110bc1e948178beda1475778f9b7462bff67 authored over 10 years ago

Merge branch 'develop'

e64554f4655f08135693cab8903b641b4d23c2a8 authored over 10 years ago

changelog: Update latest to 0.36.3.

[ci skip]

7a20f9e7aec6f14497e3687ede534974a9e750b1 authored over 10 years ago

database: Use synchronous=NORMAL.

Closes chfoo/wpull#144

f13bf4674c52f0801a73d352a1b01b78b02a50ba authored over 10 years ago

WIP Fix up .http.redirect unit tests

[ci skip]

5acec712f26fa1a30ebabd3a083a3ae92cdba7cc authored over 10 years ago

WIP .http.client: Populate requests objects

170e4a125069d74c87dc08c7c72a21c32f809d91 authored over 10 years ago

WIP Fix up .http.client.Client recorder and add unit tests

[ci skip]

9f8335a5410537fd844be3a544bf1930eb6738a6 authored over 10 years ago

WIP Fix up connection pool unit tests.

[ci skip]

362648aefd5f664ae2ddd0835977e2d36a94f1c2 authored over 10 years ago

Add testing/test.pem

[ci skip]

ff82a0f2cdd0af72cb0f57ebc3858c5743aa0661 authored over 10 years ago

WIP Fix up http Client unit test

[ci skip]

d328e0ff4b27628f6d729d0e0271c3b3b43b9f47 authored over 10 years ago

WIP Fix up SSL stream unit tests

[ci skip]

e8565fa0cab78de692d54a20458db683363eef3f authored over 10 years ago

WIP Fix up .http.request unit tests

[ci skip]

d226127edb6f64fa180d7270f5e0c99706f7ac94 authored over 10 years ago

WIP .http.stream: Wire in connection close logic.

[ci skip]

41988c63723156515aa5455f6344ca08b928f0d0 authored over 10 years ago

Merge branch 'develop' into topic/asyncio

[ci skip]

634cc7e8c19c865110aa2f171805f595a043895f authored over 10 years ago

WIP Fix up .http.stream and unit tests.

[ci skip]

9bf89b652fcc7621115f8c869498bbb1ceeba5cf authored over 10 years ago

WIP better .http.request Request objects

[ci skip]

0e9b836d2a3d4eeaa0f0f8ba45b48184389b631d authored over 10 years ago

WIP Move Body into new body module.

aa0e96936d88cbba2488c42aa4d653db91e3bccd authored over 10 years ago

Bump version to 0.36.2.

85a3909941548208f44da3faa07e638d99a5eadd authored over 10 years ago

Merge branch 'develop'

84d485a39aef239ba0d4662ed1744e65c1a344b8 authored over 10 years ago

changelog.txt: Update latest to 0.36.2.

[ci skip]

7b4ee6621ac4848f7f507c034ed90dbf6d853906 authored over 10 years ago

requirements.txt: Set tornado>=3.1.1,<4.0

a2ee9d3ddc0790190ba1c1f30a38194790db8a41 authored over 10 years ago

Bump version to 0.36.1

Merge branch 'develop'

Conflicts:
wpull/version.py

76d5021267257c58718256255f65839779701082 authored over 10 years ago

changelog: Update latest to 0.36.1.

3e33e84eef25bd379549f6f797ebb8d3708ccc0f authored over 10 years ago

Close Processor and URLTable only after stats are printed.

Closes chfoo/wpull#139.

9efb53a2be6508fa5c330084292508af33ec3df6 authored over 10 years ago

WIP Rewrite .http.client with asyncio and .http.stream.

[ci skip]

ac455d1e786eba52e42b1fab9b0e728a869b4cfe authored over 10 years ago

WIP Move RedirectTracker to .http.redirect

[ci skip]

9faf4074f304dd7780d794a14c990913de335157 authored over 10 years ago

WIP Add HTTP stream abstraction. Remove observer from chunked reader.

86491d98e0418350a73a5c13184fdca4e69adafd authored over 10 years ago

WIP Move HTTP chunked transfer reader to chunked module.

[ci skip]

34962afa52121188df57ee8ccd6618e3da35f1de authored over 10 years ago

Add contributing file.

[ci skip]

e52074773dcd90fc96bc159fff932de5a52cf643 authored over 10 years ago

Add WIP connection module for async stream abstraction

[ci skip]

08dd3a246a4a2df24efe6ab520a6723e05fb4332 authored over 10 years ago

WIP Fix up dns.py.

[ci skip]

2d0dafbfb2907ada92b2a4030dbbad447e5ba8f1 authored over 10 years ago

Merge commit '54e0b9ec47694f641e2d59f2e305c47553346034' into topic/asyncio

Conflicts:
wpull/phantomjs.py
wpull/processor.py

[ci skip]

d00521df84b01a5b010538b4b5cc56af79449691 authored over 10 years ago

Bump version to 0.36.

Merge commit '7f47914ff781dccfee18ecf3fde966a2809f1b69'

Conflicts:
wpull/version.py

6d8c25544f6517d830cc11754da4fe434687531e authored over 10 years ago

Bump version to 0.37a1.

54e0b9ec47694f641e2d59f2e305c47553346034 authored over 10 years ago

changelog: Update latest to 0.36.

[ci skip]

7f47914ff781dccfee18ecf3fde966a2809f1b69 authored over 10 years ago

scraper.py: Add srcset link scrape.

Closes chfoo/wpull#134

32d9354eb82b75e485ab1cc3602f49961f7261b5 authored over 10 years ago

observer: Allow handlers to be removed during notify.

Re: https://github.com/chfoo/wpull/commit/b12d77dd551c640a1f99c6ca4ac549e16dd6f533#commitcomment...

a862e8cc32c3aae6e99aac7f639f807f854f6491 authored over 10 years ago

Add --phantomjs-exe option.

d376ec0fa45aff12c33bc3a689deff9f396a56d5 authored over 10 years ago

processor.py: Workaround PhantomJS timeouts.

Re: chfoo/wpull#137

01ae1b40cbb9b4d1ea091086d6b795bafff52a1a authored over 10 years ago

Move stop callback concerns from Engine to Application.

ba60ec29414d5942c8d41b91da2890f765f5223d authored over 10 years ago

WIP Port tornado async to asyncio/trollius.

[ci skip]

078e3e31f8fc7fb9e5c1d429d4a172ef66762c79 authored over 10 years ago

phantomjs: Rename page_event→page_observer and use Observer.

99edac21331b6c35e8c3fc3c4c04895f2791ee65 authored over 10 years ago

Add observer module.

This module uses more conventional names. It will replace the 'actor'
module to avoid confusing ...

b12d77dd551c640a1f99c6ca4ac549e16dd6f533 authored over 10 years ago

Format with autopep8.

d84624051f3d56b9bf94784117027dc97b06b7cb authored over 10 years ago

Add and use BraceMessage for log formatting.

56064e249cd1ef2f68ff1960064fbcaf522e6625 authored over 10 years ago

Split 'network' module into 'bandwidth' and 'dns'

5725908cc7911a672f87d472fe4bb20621f42591 authored over 10 years ago

Move exit code logic from Engine to Application.

828e42ec7066856d42c07da178aca9435d9863ac authored over 10 years ago

Builder.build(): Return Application instead of Engine

03974c7110a833b5c38a6e740a5c6a0368aebb12 authored over 10 years ago

testing/py_hook_script_stop.py: Add missing record_info arg.

08f7991738b572be653739e360222a0468c516af authored over 10 years ago

Bump version to 0.35.

da5b791d358e3fdd95c2bee338f0729e543bb239 authored over 10 years ago

Merge commit '977a29c048a0e55b21dabea5dfb101bb84e57fc9'

384ac8f4a62f6ba80b8d2bd2d6b6e778d3a20ce7 authored over 10 years ago

Bump version to 0.36a1.

8f75e166ce694e557078cf18e25c0728203fb2e8 authored over 10 years ago