Ecosyste.ms: OpenCollective
An open API service for software projects hosted on Open Collective.
github.com/ArchiveTeam/grab-grab
https://github.com/ArchiveTeam/grab-grab
Version 20230607.01. Use GNU Wget 1.21.3-at.20230605.01 and arguments around DNS.
ded3019e1ba80169aa1e8bcdaad3460396e0e842 authored over 1 year ago by arkiver <[email protected]>
ded3019e1ba80169aa1e8bcdaad3460396e0e842 authored over 1 year ago by arkiver <[email protected]>
Version 20221231.02. Use shards for backfeed.
da617725b907f398abd14d0e87ac62bed688852e authored about 2 years ago by arkiver <[email protected]>
da617725b907f398abd14d0e87ac62bed688852e authored about 2 years ago by arkiver <[email protected]>
Version 20221231.01. Tweakblogs.net.
f62a4ce0a64ac79918a31125f8c37cf5709cd21b authored about 2 years ago by arkiver <[email protected]>
f62a4ce0a64ac79918a31125f8c37cf5709cd21b authored about 2 years ago by arkiver <[email protected]>
Version 20221219.01. Support geolog.mydns.jp.
3dd6d5ad4d905e2d7d7ab3552f384dd7759fafe4 authored about 2 years ago by arkiver <[email protected]>
3dd6d5ad4d905e2d7d7ab3552f384dd7759fafe4 authored about 2 years ago by arkiver <[email protected]>
Version 20221210.02. Support xs4all homepages.
b6530d4f0f5963a14d9585243a080ba5242f30d2 authored about 2 years ago by arkiver <[email protected]>
b6530d4f0f5963a14d9585243a080ba5242f30d2 authored about 2 years ago by arkiver <[email protected]>
Version 20221210.01. Queue any links not matching pattern to URLs project.
abf3fe9ae2a6553eca4086f057e173ea7023354f authored about 2 years ago by arkiver <[email protected]>
abf3fe9ae2a6553eca4086f057e173ea7023354f authored about 2 years ago by arkiver <[email protected]>
Version 20221209.01. Skip URL on 423 on webryblog URL.
d6d5a02a59b81ea467cc1013b2efc6cafd7df1b3 authored about 2 years ago by arkiver <[email protected]>
d6d5a02a59b81ea467cc1013b2efc6cafd7df1b3 authored about 2 years ago by arkiver <[email protected]>
Version 20221130.01. Use one browser user-agent.
0decbd4233710ddea4cdf6739f6a9f6536c17d05 authored about 2 years ago by arkiver <[email protected]>
0decbd4233710ddea4cdf6739f6a9f6536c17d05 authored about 2 years ago by arkiver <[email protected]>
Version 20221128.01. Remove path loop detection.
d531b4592258d7241975493e9a064d28097f8b47 authored about 2 years ago by arkiver <[email protected]>
d531b4592258d7241975493e9a064d28097f8b47 authored about 2 years ago by arkiver <[email protected]>
Version 20221126.03. Queue outlinks to URLs.
79d14b110252814cf09f05a67d646bb5a1917621 authored about 2 years ago by arkiver <[email protected]>
79d14b110252814cf09f05a67d646bb5a1917621 authored about 2 years ago by arkiver <[email protected]>
Version 20221126.02. Use temporary grabtemp20221126 for webryblog.
5e1b5e2ed588630e5dbc5efb65820664e1ddefb3 authored about 2 years ago by arkiver <[email protected]>
5e1b5e2ed588630e5dbc5efb65820664e1ddefb3 authored about 2 years ago by arkiver <[email protected]>
Version 20220931.05 (yeah). Queue URLs to URLs project is not from supported domains.
b32c8f80bf4b3ab3eb207c84013b2d2ab842621a authored over 2 years ago by arkiver <[email protected]>
b32c8f80bf4b3ab3eb207c84013b2d2ab842621a authored over 2 years ago by arkiver <[email protected]>
Version 20220931.04 (yeah). Multi items size 40.
36024a9fb9aadbdac8dabc21edf57f26a4f25113 authored over 2 years ago by arkiver <[email protected]>
36024a9fb9aadbdac8dabc21edf57f26a4f25113 authored over 2 years ago by arkiver <[email protected]>
Version 20220931.03. Temporarily switch to tracker grabtemp20220831.
1034c1fdcbcc3a9025f4bc4f8ccaba1a6d3508d2 authored over 2 years ago by arkiver <[email protected]>
1034c1fdcbcc3a9025f4bc4f8ccaba1a6d3508d2 authored over 2 years ago by arkiver <[email protected]>
Update pipeline.py
50dae2e987668f49244a167e56dd629f3c1e4fe5 authored over 2 years ago by HarryC145 <[email protected]>
50dae2e987668f49244a167e56dd629f3c1e4fe5 authored over 2 years ago by HarryC145 <[email protected]>
Update pipeline.py
d669f992532c5e20074751f8d1d2c59d885fba95 authored over 2 years ago by HarryC145 <[email protected]>
d669f992532c5e20074751f8d1d2c59d885fba95 authored over 2 years ago by HarryC145 <[email protected]>
Update pipeline.py
8eb008b3a11295b1e649c293d3b0b1c3fdffc556 authored over 2 years ago by HarryC145 <[email protected]>
8eb008b3a11295b1e649c293d3b0b1c3fdffc556 authored over 2 years ago by HarryC145 <[email protected]>
Merge pull request #1 from HarryC145/patch-1
Update patterns.txt
9ff7e506f9eb1d47b848bc30da96a9fde05fe377 authored over 2 years ago by km09 <[email protected]>
Update patterns.txt
e6cfa559f876f29460b6ec6a791c31aafba85016 authored over 2 years ago by HarryC145 <[email protected]>
e6cfa559f876f29460b6ec6a791c31aafba85016 authored over 2 years ago by HarryC145 <[email protected]>
Version 20220605.01. Support GNU Wget 1.21.3-at.20220503.02. Fix killing crawl when items cannot be queued.
ce01e7dc405a9052a57500656431fbb3bf22557d authored over 2 years ago by arkiver <[email protected]>
ce01e7dc405a9052a57500656431fbb3bf22557d authored over 2 years ago by arkiver <[email protected]>
Version 20220329.04. Add more patterns for 2style domains.
71a40288109c429dad668a21b06a6f588b12f4a6 authored almost 3 years ago by arkiver <[email protected]>
71a40288109c429dad668a21b06a6f588b12f4a6 authored almost 3 years ago by arkiver <[email protected]>
Version 20220329.03. Add patterns for 2style domains.
32ae30a66b2a30812e8a21876a36ee7146e83813 authored almost 3 years ago by arkiver <[email protected]>
32ae30a66b2a30812e8a21876a36ee7146e83813 authored almost 3 years ago by arkiver <[email protected]>
Version 20220329.02. - to %- in patterns.
b2259bc58f65e33611229d40dad178a4d47d20f9 authored almost 3 years ago by arkiver <[email protected]>
b2259bc58f65e33611229d40dad178a4d47d20f9 authored almost 3 years ago by arkiver <[email protected]>
Version 20220329.01. Prepare for various webcrow domains.
62fb44ef400271846c693810f1bde002707b0d31 authored almost 3 years ago by arkiver <[email protected]>
62fb44ef400271846c693810f1bde002707b0d31 authored almost 3 years ago by arkiver <[email protected]>
Version 20220327.02. Ignore bad extracted URLs.
9cc275efdd2b9a460eedea81a7782bc2035e3073 authored almost 3 years ago by arkiver <[email protected]>
9cc275efdd2b9a460eedea81a7782bc2035e3073 authored almost 3 years ago by arkiver <[email protected]>
Version 20220327.01. Support ria-m.tv.
0c95d91e8e1c11e85b59b4b12999d5d8547565a5 authored almost 3 years ago by arkiver <[email protected]>
0c95d91e8e1c11e85b59b4b12999d5d8547565a5 authored almost 3 years ago by arkiver <[email protected]>
Version 20220323.05. Support queuing large number of URLs.
239ba454381678280e40be81233e128a1223332e authored almost 3 years ago by arkiver <[email protected]>
239ba454381678280e40be81233e128a1223332e authored almost 3 years ago by arkiver <[email protected]>
Version 20220323.04. Handle large number of discovered URLs. Fix maxtries usage.
b5699f07787945fffc024131279d3bd2dcdc3c69 authored almost 3 years ago by arkiver <[email protected]>
b5699f07787945fffc024131279d3bd2dcdc3c69 authored almost 3 years ago by arkiver <[email protected]>
Version 20220323.03. Add stackoverflow.com/jobs.
9c17605e7322ba98c67cd044fa814a3b293b9dc3 authored almost 3 years ago by arkiver <[email protected]>
9c17605e7322ba98c67cd044fa814a3b293b9dc3 authored almost 3 years ago by arkiver <[email protected]>
Version 20220323.02. Support joindota.com.
028dd58298371a0780eaebadea65e0439f0f493a authored almost 3 years ago by arkiver <[email protected]>
028dd58298371a0780eaebadea65e0439f0f493a authored almost 3 years ago by arkiver <[email protected]>
Version 20220323.01. Support vfl.ru.
eef49c0ca9581162a81b6979af1213ff08d4e5d5 authored almost 3 years ago by arkiver <[email protected]>
eef49c0ca9581162a81b6979af1213ff08d4e5d5 authored almost 3 years ago by arkiver <[email protected]>
Version 20220318.04. Handle meta.org.
6f3263956094ea53144380271136529ab09bac34 authored almost 3 years ago by arkiver <[email protected]>
6f3263956094ea53144380271136529ab09bac34 authored almost 3 years ago by arkiver <[email protected]>
Version 20220318.03.
0195d71c6fb948ac88d54faf5bcd7b5d74a4683c authored almost 3 years ago by arkiver <[email protected]>
0195d71c6fb948ac88d54faf5bcd7b5d74a4683c authored almost 3 years ago by arkiver <[email protected]>
Version 20220318.02. Support dslreports.com.
891bdbd16ccc0d0d3faff96a26058084d8482657 authored almost 3 years ago by arkiver <[email protected]>
891bdbd16ccc0d0d3faff96a26058084d8482657 authored almost 3 years ago by arkiver <[email protected]>
Version 20220318.02. Support dslreports.com.
b666ed7f7748a2a00d0d48196173cd9925f939b8 authored almost 3 years ago by arkiver <[email protected]>
b666ed7f7748a2a00d0d48196173cd9925f939b8 authored almost 3 years ago by arkiver <[email protected]>
Version 20220318.02. Support dslreports.com.
de8301ee14ecb641a85131d507f637b76f8a0ca1 authored almost 3 years ago by arkiver <[email protected]>
de8301ee14ecb641a85131d507f637b76f8a0ca1 authored almost 3 years ago by arkiver <[email protected]>
initial
e903d95967ed692aff1ce76ef50622c0c417c7d6 authored almost 3 years ago by arkiver <[email protected]>
e903d95967ed692aff1ce76ef50622c0c417c7d6 authored almost 3 years ago by arkiver <[email protected]>