Ecosyste.ms: OpenCollective
An open API service for software projects hosted on Open Collective.
github.com/ArchiveTeam/grab-site
The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns
https://github.com/ArchiveTeam/grab-site
README: Tweak
3f295d7ecb19f09320a4d2ef311c9a6bb4d4406c authored about 9 years ago by Ivan Kozik <[email protected]>
3f295d7ecb19f09320a4d2ef311c9a6bb4d4406c authored about 9 years ago by Ivan Kozik <[email protected]>
Document how to use webrecorder.io instead of grab-site for archiving twitter.com/user pages
1dc6cac6d83956525cdb95b247eaf5a70afa87c0 authored about 9 years ago by Ivan Kozik <[email protected]>
1dc6cac6d83956525cdb95b247eaf5a70afa87c0 authored about 9 years ago by Ivan Kozik <[email protected]>
Bump version
01bf6d527bc1556e77f20218c11f8fbfa771cf90 authored about 9 years ago by Ivan Kozik <[email protected]>
01bf6d527bc1556e77f20218c11f8fbfa771cf90 authored about 9 years ago by Ivan Kozik <[email protected]>
Merge pull request #67 from DanielOaks/fix-ws-exception
server: Fix exception when clients visit ws port via a browser
64d928b82e09316fb2b683a2aebc5b12727b6462 authored about 9 years ago by Ivan Kozik <[email protected]>
server: Fix exception when clients visit ws port via a browser
04179a2a9934b2bd5897bdcbd635f2638f504f00 authored about 9 years ago by Daniel Oaks <[email protected]>
04179a2a9934b2bd5897bdcbd635f2638f504f00 authored about 9 years ago by Daniel Oaks <[email protected]>
global igset: ignore two more loops
111ffca64333498113f7177e69463f3a69afbb42 authored about 9 years ago by Ivan Kozik <[email protected]>
111ffca64333498113f7177e69463f3a69afbb42 authored about 9 years ago by Ivan Kozik <[email protected]>
global igset: ignore livejournal.com/identity/login.bml
5f14263070e31cb43a7fe3621b7f16048b5ab774 authored about 9 years ago by Ivan Kozik <[email protected]>
5f14263070e31cb43a7fe3621b7f16048b5ab774 authored about 9 years ago by Ivan Kozik <[email protected]>
README: Upgrade step is not quite right for pyenv users
7be4017347b3a3c36d18258ef2836b52a6413e38 authored about 9 years ago by Ivan Kozik <[email protected]>
7be4017347b3a3c36d18258ef2836b52a6413e38 authored about 9 years ago by Ivan Kozik <[email protected]>
README: Add ~/.local/bin/ instructions here too
6e9fd55463ee9a9a2fa6f5a84af64c0c4b3d2130 authored about 9 years ago by Ivan Kozik <[email protected]>
6e9fd55463ee9a9a2fa6f5a84af64c0c4b3d2130 authored about 9 years ago by Ivan Kozik <[email protected]>
README: Tweak to fix markdown rendering?
c78ec516dce5ced0598957e92e1929195a8216c2 authored about 9 years ago by Ivan Kozik <[email protected]>
c78ec516dce5ced0598957e92e1929195a8216c2 authored about 9 years ago by Ivan Kozik <[email protected]>
README: Tweak to fix markdown rendering?
5edd79c5471490f27babdbf6a4694e55b42ebd53 authored about 9 years ago by Ivan Kozik <[email protected]>
5edd79c5471490f27babdbf6a4694e55b42ebd53 authored about 9 years ago by Ivan Kozik <[email protected]>
README: Tweak to fix markdown rendering?
7a928ae7fcc3aa4ed1b6bd5a902cf349c7476a49 authored about 9 years ago by Ivan Kozik <[email protected]>
7a928ae7fcc3aa4ed1b6bd5a902cf349c7476a49 authored about 9 years ago by Ivan Kozik <[email protected]>
README: Update TOC
ccdad68099652a834a0ea55ad097da1c149ff2bd authored about 9 years ago by Ivan Kozik <[email protected]>
ccdad68099652a834a0ea55ad097da1c149ff2bd authored about 9 years ago by Ivan Kozik <[email protected]>
README: Add pyenv instructions
a51ae254dfd83995f7a99c2b402df2fafacf1014 authored about 9 years ago by Ivan Kozik <[email protected]>
a51ae254dfd83995f7a99c2b402df2fafacf1014 authored about 9 years ago by Ivan Kozik <[email protected]>
lstrip '-' to avoid creating filenames that must be --'ed or quoted
2acc826d563a81c71ce090bcda65f6d163194849 authored about 9 years ago by Ivan Kozik <[email protected]>
2acc826d563a81c71ce090bcda65f6d163194849 authored about 9 years ago by Ivan Kozik <[email protected]>
Revert "README: Tweak"
This reverts commit dd0c005331dc182e6251823f5c90030c120680b3.
```bash coloring is unhelpfully b...
495beca32a25ad24a61f4cb1523be6d12b21c691 authored about 9 years ago by Ivan Kozik <[email protected]>
README: Tweak
dd0c005331dc182e6251823f5c90030c120680b3 authored about 9 years ago by Ivan Kozik <[email protected]>
dd0c005331dc182e6251823f5c90030c120680b3 authored about 9 years ago by Ivan Kozik <[email protected]>
Update Ubuntu 15.10 instructions now that https://bugs.launchpad.net/ubuntu/+source/python-pip/+bug/1518023 is fixed
cb0fa6591b9d976e91fe59b93150a292a96618ee authored about 9 years ago by Ivan Kozik <[email protected]>
cb0fa6591b9d976e91fe59b93150a292a96618ee authored about 9 years ago by Ivan Kozik <[email protected]>
global igset: Ignore a loop on archive.org
4ea80eec80ec49ebb4f039ce41ad7f082753f099 authored about 9 years ago by Ivan Kozik <[email protected]>
4ea80eec80ec49ebb4f039ce41ad7f082753f099 authored about 9 years ago by Ivan Kozik <[email protected]>
global igset: ignore /wp-admin/
38f733f9d2730baf6edf94e09878825aa607f163 authored about 9 years ago by Ivan Kozik <[email protected]>
38f733f9d2730baf6edf94e09878825aa607f163 authored about 9 years ago by Ivan Kozik <[email protected]>
Add --id=, --dir=, and --finished-warc-dir= options
adb35ee4e3369158f8d85421b384f953b11273a6 authored about 9 years ago by Ivan Kozik <[email protected]>
adb35ee4e3369158f8d85421b384f953b11273a6 authored about 9 years ago by Ivan Kozik <[email protected]>
README: Tweak
17464e7b740035d1cc00177a2658da6adfdaf002 authored about 9 years ago by Ivan Kozik <[email protected]>
17464e7b740035d1cc00177a2658da6adfdaf002 authored about 9 years ago by Ivan Kozik <[email protected]>
Set NCR=1 cookie for .blogspot.com to avoid getting redirected
f8fece9ebbbc2c161867418c6d48da3e6d83355d authored about 9 years ago by Ivan Kozik <[email protected]>
f8fece9ebbbc2c161867418c6d48da3e6d83355d authored about 9 years ago by Ivan Kozik <[email protected]>
README: Add travis link and shields.io button
7975566add3b82fc86ee889902235ccabffe36c6 authored about 9 years ago by Ivan Kozik <[email protected]>
7975566add3b82fc86ee889902235ccabffe36c6 authored about 9 years ago by Ivan Kozik <[email protected]>
.travis.yml: Test a few more things
0e5f8c8b83605dc3f4a8feeda2cc33a3bcc083f5 authored about 9 years ago by Ivan Kozik <[email protected]>
0e5f8c8b83605dc3f4a8feeda2cc33a3bcc083f5 authored about 9 years ago by Ivan Kozik <[email protected]>
.travis.yml: Remove osx until https://github.com/travis-ci/travis-ci/issues/2312 is fixed
9c3c3c3afdaeb0acecdcfa6d303eae18bc6596b7 authored about 9 years ago by Ivan Kozik <[email protected]>
9c3c3c3afdaeb0acecdcfa6d303eae18bc6596b7 authored about 9 years ago by Ivan Kozik <[email protected]>
.travis.yml: pip3 install verbosely
e023105c5ebd88b86faa473f6323192c3918dd14 authored about 9 years ago by Ivan Kozik <[email protected]>
e023105c5ebd88b86faa473f6323192c3918dd14 authored about 9 years ago by Ivan Kozik <[email protected]>
.travis.yml: Fix script
bdf16848343efb068c3ae66f9b9f7bc1ddfbcc23 authored about 9 years ago by Ivan Kozik <[email protected]>
bdf16848343efb068c3ae66f9b9f7bc1ddfbcc23 authored about 9 years ago by Ivan Kozik <[email protected]>
.travis.yml: remove --user because we're in a virtualenv
37ddc8c6b88f2f1a225d34fcd275d9e3e4f42b3f authored about 9 years ago by Ivan Kozik <[email protected]>
37ddc8c6b88f2f1a225d34fcd275d9e3e4f42b3f authored about 9 years ago by Ivan Kozik <[email protected]>
Add a .travis.yml
2f121a864be6e9d871ec17c9938942792ac30cde authored about 9 years ago by Ivan Kozik <[email protected]>
2f121a864be6e9d871ec17c9938942792ac30cde authored about 9 years ago by Ivan Kozik <[email protected]>
Send an over18=1 cookie to reddit.com to avoid the age gate on many subreddits
86e92d684c19f0f4097b896374d37eea2137b661 authored about 9 years ago by Ivan Kozik <[email protected]>
86e92d684c19f0f4097b896374d37eea2137b661 authored about 9 years ago by Ivan Kozik <[email protected]>
If using Python 3.4.0, depend on an older version of aiohttp that works on Python 3.4.0
6a647f637e9ab00e63ffd6dbfaff3ca34f6caeb3 authored about 9 years ago by Ivan Kozik <[email protected]>
6a647f637e9ab00e63ffd6dbfaff3ca34f6caeb3 authored about 9 years ago by Ivan Kozik <[email protected]>
README: Tweak
d96809ea785d41f488553e51fb962f8171dfe851 authored about 9 years ago by Ivan Kozik <[email protected]>
d96809ea785d41f488553e51fb962f8171dfe851 authored about 9 years ago by Ivan Kozik <[email protected]>
README: More subreddit crawling
a9a06127eae19c1b1a9d59603074a7b62459a22d authored about 9 years ago by Ivan Kozik <[email protected]>
a9a06127eae19c1b1a9d59603074a7b62459a22d authored about 9 years ago by Ivan Kozik <[email protected]>
README: Tweak
01a72b910a372050e29d70510a72064fee244628 authored about 9 years ago by Ivan Kozik <[email protected]>
01a72b910a372050e29d70510a72064fee244628 authored about 9 years ago by Ivan Kozik <[email protected]>
README: Document how to archive websites whose domains have just expired
a2f1192b9212742cd5a18f4abb358f602632b221 authored about 9 years ago by Ivan Kozik <[email protected]>
a2f1192b9212742cd5a18f4abb358f602632b221 authored about 9 years ago by Ivan Kozik <[email protected]>
README: Document how to archive subreddits
0363f825d506c71e7db383333d44fba37cda3302 authored about 9 years ago by Ivan Kozik <[email protected]>
0363f825d506c71e7db383333d44fba37cda3302 authored about 9 years ago by Ivan Kozik <[email protected]>
README: Tweak
52895892d58386892d85b70b9ff8f96102b5a421 authored about 9 years ago by Ivan Kozik <[email protected]>
52895892d58386892d85b70b9ff8f96102b5a421 authored about 9 years ago by Ivan Kozik <[email protected]>
README: Tweak
e82937a9f6b51165263cafe79c7bc9cf4d1cab56 authored about 9 years ago by Ivan Kozik <[email protected]>
e82937a9f6b51165263cafe79c7bc9cf4d1cab56 authored about 9 years ago by Ivan Kozik <[email protected]>
README: Fix headings
6d3f0f901d0fc3dd3eb4e73eb30540520ac42db1 authored about 9 years ago by Ivan Kozik <[email protected]>
6d3f0f901d0fc3dd3eb4e73eb30540520ac42db1 authored about 9 years ago by Ivan Kozik <[email protected]>
README: Write about how to archive specific websites
140efe082ae83f0999c486fda5cc0087c7c986cb authored about 9 years ago by Ivan Kozik <[email protected]>
140efe082ae83f0999c486fda5cc0087c7c986cb authored about 9 years ago by Ivan Kozik <[email protected]>
README: Tweak
b000c1a7595931b10e0da0d4d848a47709e5eb37 authored about 9 years ago by Ivan Kozik <[email protected]>
b000c1a7595931b10e0da0d4d848a47709e5eb37 authored about 9 years ago by Ivan Kozik <[email protected]>
Bump Firefox version in UA string
8a6eaea16c06fb83c5312d34ef1798f19acf4c3c authored about 9 years ago by Ivan Kozik <[email protected]>
8a6eaea16c06fb83c5312d34ef1798f19acf4c3c authored about 9 years ago by Ivan Kozik <[email protected]>
Add warning about warcprox
d80bde8c8cfbca2b10bde74e8dee1ffc4d02a36b authored about 9 years ago by Ivan Kozik <[email protected]>
d80bde8c8cfbca2b10bde74e8dee1ffc4d02a36b authored about 9 years ago by Ivan Kozik <[email protected]>
Use Roboto font if installed
58a2711058ab7b99a95c497cee35c6c2bfeffd98 authored about 9 years ago by Ivan Kozik <[email protected]>
58a2711058ab7b99a95c497cee35c6c2bfeffd98 authored about 9 years ago by Ivan Kozik <[email protected]>
Don't crash if psutil is not available on non-Windows OS (it is no longer installed by wpull 1.2.2)
e72c5fc3a79d04332416142661c934ceccd85055 authored about 9 years ago by Ivan Kozik <[email protected]>
e72c5fc3a79d04332416142661c934ceccd85055 authored about 9 years ago by Ivan Kozik <[email protected]>
Don't use --debug-manhole on Windows to avoid a crash
File "C:\Python34\lib\site-packages\click\core.py", line 680, in main
rv = self.invoke(ctx...
Don't crash on lack of SIGINT support on Windows
ca5c4611faf00d4e414f55ebc9f098de5a116e65 authored about 9 years ago by Ivan Kozik <[email protected]>
ca5c4611faf00d4e414f55ebc9f098de5a116e65 authored about 9 years ago by Ivan Kozik <[email protected]>
dupes: Also catch lmdb.Error for Windows, which complains about lacking disk space
7a0c1b73bcb69c9a33c764868f2bb4c64a79c615 authored about 9 years ago by Ivan Kozik <[email protected]>
7a0c1b73bcb69c9a33c764868f2bb4c64a79c615 authored about 9 years ago by Ivan Kozik <[email protected]>
Don't try to use the unavailable --monitor- options on Windows
e6ebbd7b5166b4878858228d382cb8be9128fc08 authored about 9 years ago by Ivan Kozik <[email protected]>
e6ebbd7b5166b4878858228d382cb8be9128fc08 authored about 9 years ago by Ivan Kozik <[email protected]>
setup.py: if GRAB_SITE_NO_CCHARDET env var set, don't require cchardet; wpull will fall back on chardet
ec9f7bdb438d521551978a18208c772e81e394c7 authored about 9 years ago by Ivan Kozik <[email protected]>
ec9f7bdb438d521551978a18208c772e81e394c7 authored about 9 years ago by Ivan Kozik <[email protected]>
README: Tweak
f00ab849496789c8360397431ac79256a9848a56 authored about 9 years ago by Ivan Kozik <[email protected]>
f00ab849496789c8360397431ac79256a9848a56 authored about 9 years ago by Ivan Kozik <[email protected]>
Clarify upgrade instructions
8dc43fe73c178bbfa71f0a1e2c276e2f97724061 authored about 9 years ago by Ivan Kozik <[email protected]>
8dc43fe73c178bbfa71f0a1e2c276e2f97724061 authored about 9 years ago by Ivan Kozik <[email protected]>
global igset: also ignore recaptcha /mailhide/d links
1bb8bcc4d8af0e12ee666fa5544e6ec6a062352d authored about 9 years ago by Ivan Kozik <[email protected]>
1bb8bcc4d8af0e12ee666fa5544e6ec6a062352d authored about 9 years ago by Ivan Kozik <[email protected]>
global igset: Ignore /impixu on a new tumblr domain
40ca80638d76aa56b6376ff07dd5baf221567293 authored about 9 years ago by Ivan Kozik <[email protected]>
40ca80638d76aa56b6376ff07dd5baf221567293 authored about 9 years ago by Ivan Kozik <[email protected]>
Add Ubuntu 15.10 install instructions
6ac6bf69d466d43ea077a9d875515ca630702abf authored about 9 years ago by Ivan Kozik <[email protected]>
6ac6bf69d466d43ea077a9d875515ca630702abf authored about 9 years ago by Ivan Kozik <[email protected]>
Use wpull>=1.2.2
4487c43c8359f222692e199c2f41d0d86a1cfd36 authored about 9 years ago by Ivan Kozik <[email protected]>
4487c43c8359f222692e199c2f41d0d86a1cfd36 authored about 9 years ago by Ivan Kozik <[email protected]>
README: Update TOC
6da093a8176b3f5e87f660893aa696196e2e4e6d authored about 9 years ago by Ivan Kozik <[email protected]>
6da093a8176b3f5e87f660893aa696196e2e4e6d authored about 9 years ago by Ivan Kozik <[email protected]>
README: Remove instructions for grabbing a site that requires a cookie because the --header solution results in the cookie being sent to all domains instead of just the intended domain
3dfb9dc757ca182b514d076483bd71cb0b9f41bf authored about 9 years ago by Ivan Kozik <[email protected]>
3dfb9dc757ca182b514d076483bd71cb0b9f41bf authored about 9 years ago by Ivan Kozik <[email protected]>
README: Be more specific about which Python 3 must be installed
77248eb47c3a483f34e5fa9cbd729df058bebb4d authored about 9 years ago by Ivan Kozik <[email protected]>
77248eb47c3a483f34e5fa9cbd729df058bebb4d authored about 9 years ago by Ivan Kozik <[email protected]>
README: OS X install also works on 10.11
8101d4b04c9037ef702590143f4104d7c1526f10 authored about 9 years ago by Ivan Kozik <[email protected]>
8101d4b04c9037ef702590143f4104d7c1526f10 authored about 9 years ago by Ivan Kozik <[email protected]>
Add a table of contents
9f3fe3f4cb257f0cdb34669962456fdfc9ba7030 authored about 9 years ago by Ivan Kozik <[email protected]>
9f3fe3f4cb257f0cdb34669962456fdfc9ba7030 authored about 9 years ago by Ivan Kozik <[email protected]>
Document zless and how to grab a site that requires a cookie
74c385332a1e0ab558e20cd7a7775cbf6b73bb36 authored about 9 years ago by Ivan Kozik <[email protected]>
74c385332a1e0ab558e20cd7a7775cbf6b73bb36 authored about 9 years ago by Ivan Kozik <[email protected]>
Mention --no-skip-getaddrinfo
19571543216d612b5def8fd1c034e274eea75437 authored over 9 years ago by Ivan Kozik <[email protected]>
19571543216d612b5def8fd1c034e274eea75437 authored over 9 years ago by Ivan Kozik <[email protected]>
Fix: new Click gives us () instead of None when no start_url's are given
b3c433a60b4ce044309d5a5a0ba126cba6be4566 authored over 9 years ago by Ivan Kozik <[email protected]>
b3c433a60b4ce044309d5a5a0ba126cba6be4566 authored over 9 years ago by Ivan Kozik <[email protected]>
README: Tweak
ea483bc74b04b4a69a42a817cbb53576edd4590d authored over 9 years ago by Ivan Kozik <[email protected]>
ea483bc74b04b4a69a42a817cbb53576edd4590d authored over 9 years ago by Ivan Kozik <[email protected]>
Add note about Python 3.5
8260a0d7cf10ee6cd6faf5792f29926fa9374441 authored over 9 years ago by Ivan Kozik <[email protected]>
8260a0d7cf10ee6cd6faf5792f29926fa9374441 authored over 9 years ago by Ivan Kozik <[email protected]>
Document how to upgrade grab-site
bfe92c9605ae8ab7e40a563c553322258c833090 authored over 9 years ago by Ivan Kozik <[email protected]>
bfe92c9605ae8ab7e40a563c553322258c833090 authored over 9 years ago by Ivan Kozik <[email protected]>
Add --no-dupespotter for turning off dupespotter which sometimes has false positives
7a63a3dcd113e11de218fe6bb0c3ad03153a6954 authored over 9 years ago by Ivan Kozik <[email protected]>
7a63a3dcd113e11de218fe6bb0c3ad03153a6954 authored over 9 years ago by Ivan Kozik <[email protected]>
Update default user agent
17c1b9caaac84084d4ae7c2829f74cd852db5445 authored over 9 years ago by Ivan Kozik <[email protected]>
17c1b9caaac84084d4ae7c2829f74cd852db5445 authored over 9 years ago by Ivan Kozik <[email protected]>
Write URLs skipped by --max-content-length= to DIR/skipped_max_content_length
f1548521ec57ab6dd4eaec5f7c0c637fa932d2ce authored over 9 years ago by Ivan Kozik <[email protected]>
f1548521ec57ab6dd4eaec5f7c0c637fa932d2ce authored over 9 years ago by Ivan Kozik <[email protected]>
Fix for 32-bit machines: don't crash on startup with lmdb.MemoryError
lmdb.MemoryError: [...]/dupes_db: Cannot allocate memory
3def2a79bcc69602b3ca5247c52f2a522d0d29c1 authored over 9 years ago by Ivan Kozik <[email protected]>
Bump version
e0ad2e9a25ffe019216ee557b277a524fffdc517 authored over 9 years ago by Ivan Kozik <[email protected]>
e0ad2e9a25ffe019216ee557b277a524fffdc517 authored over 9 years ago by Ivan Kozik <[email protected]>
Prevent twitter crawls from endlessly downloading [\?&]nav=
Credit to garyrh
5a04a38f59080d375c6f8c5c05a53266fd418964 authored over 9 years ago by Ivan Kozik <[email protected]>
Explain imdb ignore set
c28c593a839ee09e067f91b26aa82d83ce8a30db authored over 9 years ago by Ivan Kozik <[email protected]>
c28c593a839ee09e067f91b26aa82d83ce8a30db authored over 9 years ago by Ivan Kozik <[email protected]>
Add --no-video option to skip the download of videos
b782c233891a170f531efbcfbe4e9c5d4fc3163a authored over 9 years ago by Ivan Kozik <[email protected]>
b782c233891a170f531efbcfbe4e9c5d4fc3163a authored over 9 years ago by Ivan Kozik <[email protected]>
Add --warc-max-size=BYTES option for controlling WARC size
6f6754f81e66c63b7e44d351e1f3513a09b25a9b authored over 9 years ago by Arkiver2 <[email protected]>
6f6754f81e66c63b7e44d351e1f3513a09b25a9b authored over 9 years ago by Arkiver2 <[email protected]>
Add support for passing multiple URLs to grab-site
ee2684941d8367ab6e8571f4c5d7e6dcc237dc9d authored over 9 years ago by Ivan Kozik <[email protected]>
ee2684941d8367ab6e8571f4c5d7e6dcc237dc9d authored over 9 years ago by Ivan Kozik <[email protected]>
Fix blogspot search? ignore
524cdf2cecae7f5b7e928fe3697d46b2bd8b9aa7 authored over 9 years ago by Ivan Kozik <[email protected]>
524cdf2cecae7f5b7e928fe3697d46b2bd8b9aa7 authored over 9 years ago by Ivan Kozik <[email protected]>
Don't crash on --igsets=blogs even though it's gone
f379264ed175d63f953c178fc750ee42dfda0ab3 authored over 9 years ago by Ivan Kozik <[email protected]>
f379264ed175d63f953c178fc750ee42dfda0ab3 authored over 9 years ago by Ivan Kozik <[email protected]>
Remove mentions of blogs ignore set
49aa6d0dcf6e69567c3f3cee9e72e6e4d90f4c2a authored over 9 years ago by Ivan Kozik <[email protected]>
49aa6d0dcf6e69567c3f3cee9e72e6e4d90f4c2a authored over 9 years ago by Ivan Kozik <[email protected]>
Add comment to reddit ignore set
6a6dff0083691d7cf1f02427d3290b4afe3e4639 authored over 9 years ago by Ivan Kozik <[email protected]>
6a6dff0083691d7cf1f02427d3290b4afe3e4639 authored over 9 years ago by Ivan Kozik <[email protected]>
Migrate all other ignores from blogs to the global set
4b50c6de67326a43c0f89ede1fc324bb4127634c authored over 9 years ago by Ivan Kozik <[email protected]>
4b50c6de67326a43c0f89ede1fc324bb4127634c authored over 9 years ago by Ivan Kozik <[email protected]>
Remove ignores that probably only wget needed
954ab31acb1e0979d035c3dbdd25687d4ba5e48a authored over 9 years ago by Ivan Kozik <[email protected]>
954ab31acb1e0979d035c3dbdd25687d4ba5e48a authored over 9 years ago by Ivan Kozik <[email protected]>
Migrate some ignores from blogs to global set
b83bce1f2a4ca03d005538753dfdc54d18f8788a authored over 9 years ago by Ivan Kozik <[email protected]>
b83bce1f2a4ca03d005538753dfdc54d18f8788a authored over 9 years ago by Ivan Kozik <[email protected]>
Ignore per-post and per-comment Atom feeds on blogspot.com
1b8b4b00778b278415aabdd61ed0f841d77cd051 authored over 9 years ago by Ivan Kozik <[email protected]>
1b8b4b00778b278415aabdd61ed0f841d77cd051 authored over 9 years ago by Ivan Kozik <[email protected]>
Fix: --offsite-links should be on by default
291b3e939bd70049cd5ff39eb8dbb7f6678da839 authored over 9 years ago by Ivan Kozik <[email protected]>
291b3e939bd70049cd5ff39eb8dbb7f6678da839 authored over 9 years ago by Ivan Kozik <[email protected]>
Cache control files for just 1.5 sec instead of 3 sec
a3f1ff7ed901b7014da34f7631913b3dece37abd authored over 9 years ago by Ivan Kozik <[email protected]>
a3f1ff7ed901b7014da34f7631913b3dece37abd authored over 9 years ago by Ivan Kozik <[email protected]>
Fix very recent regression: report the pattern instead of the regexp
050dbc44d81d526906f2c1fa9b46edce9e41950e authored over 9 years ago by Ivan Kozik <[email protected]>
050dbc44d81d526906f2c1fa9b46edce9e41950e authored over 9 years ago by Ivan Kozik <[email protected]>
Remove no-longer-needed workaround in ignore sets
b8b6248aab4e3148e7201ca4af6b2bee63ed9340 authored over 9 years ago by Ivan Kozik <[email protected]>
b8b6248aab4e3148e7201ca4af6b2bee63ed9340 authored over 9 years ago by Ivan Kozik <[email protected]>
Increase size of compiled regexp cache; remove unused code
1d52a28fac91adbdf2d6d69c3175b2a8879444f3 authored over 9 years ago by Ivan Kozik <[email protected]>
1d52a28fac91adbdf2d6d69c3175b2a8879444f3 authored over 9 years ago by Ivan Kozik <[email protected]>
README: tweak
13d73ad59f020f74f86bd3bc074143c08780965b authored over 9 years ago by Ivan Kozik <[email protected]>
13d73ad59f020f74f86bd3bc074143c08780965b authored over 9 years ago by Ivan Kozik <[email protected]>
Implement --wpull-args for passing additional arguments to wpull
26c7ea84d89daf3d712132afbfde34c70d8c3767 authored over 9 years ago by Ivan Kozik <[email protected]>
26c7ea84d89daf3d712132afbfde34c70d8c3767 authored over 9 years ago by Ivan Kozik <[email protected]>
Don't crash if DIR/concurrency is set to 0
1674751b1cef38b4b9e4d8d00100e7b8c70f82bd authored over 9 years ago by Ivan Kozik <[email protected]>
1674751b1cef38b4b9e4d8d00100e7b8c70f82bd authored over 9 years ago by Ivan Kozik <[email protected]>
Reorder options in README
db212f67167d5bfbecdd8e673866dfffbc3135ff authored over 9 years ago by Ivan Kozik <[email protected]>
db212f67167d5bfbecdd8e673866dfffbc3135ff authored over 9 years ago by Ivan Kozik <[email protected]>
Bump version
28f5652404f12726617496d2d0e361e5d58dbdfa authored over 9 years ago by Ivan Kozik <[email protected]>
28f5652404f12726617496d2d0e361e5d58dbdfa authored over 9 years ago by Ivan Kozik <[email protected]>
Print ignores without doubling up backslashes
ba823a34f89af59f030fe9ee1ec530e6ced8c187 authored over 9 years ago by Ivan Kozik <[email protected]>
ba823a34f89af59f030fe9ee1ec530e6ced8c187 authored over 9 years ago by Ivan Kozik <[email protected]>
Implement -i / --input-file, supporting both local input files and URLs
668c03d5d2f9e76e44bda505206f2aabd9cce3f2 authored over 9 years ago by Ivan Kozik <[email protected]>
668c03d5d2f9e76e44bda505206f2aabd9cce3f2 authored over 9 years ago by Ivan Kozik <[email protected]>
README: tweak
37a6cb655c093663c39e611ea6eb5821f0259b0f authored over 9 years ago by Ivan Kozik <[email protected]>
37a6cb655c093663c39e611ea6eb5821f0259b0f authored over 9 years ago by Ivan Kozik <[email protected]>
Pretend to be Firefox 40; it's out tomorrow
9989eb5b707b932a215bb2e4a72057df7904f9e5 authored over 9 years ago by Ivan Kozik <[email protected]>
9989eb5b707b932a215bb2e4a72057df7904f9e5 authored over 9 years ago by Ivan Kozik <[email protected]>