Ecosyste.ms: OpenCollective

An open API service for software projects hosted on Open Collective.

github.com/ArchiveTeam/NewsGrabber

Grabbing all news.
https://github.com/ArchiveTeam/NewsGrabber

Merge pull request #90 from Lagittaja/master

Add nordic news sites

2f8f6e583ce84123be03037352b28c176feef467 authored over 7 years ago by HarryC145 <[email protected]>
Add files via upload

io-tech

a108ddd2ba15f6e1d5e4ec5a005f24a032337fe2 authored over 7 years ago by Lagittaja <[email protected]>
Update web__iotech_fi.py

1f58e86126adbc050f5cc18bd3e10b0a913d0201 authored over 7 years ago by Lagittaja <[email protected]>
Add files via upload

Norway news

6200e1478fa3135ef346673f6bbc3e3147def673 authored over 7 years ago by Lagittaja <[email protected]>
Update web__gp_se.py

d3ff2a042ceb59e5543140b4c3e8b3c46d3434f9 authored over 7 years ago by Lagittaja <[email protected]>
Add files via upload

Swedish news

9fb6cf55af4dd32f850b89a6e36bf6a594e2b0f4 authored over 7 years ago by Lagittaja <[email protected]>
Add files via upload

aa85278bb8a85b2347249578bd8a600c3705988a authored over 7 years ago by Lagittaja <[email protected]>
Add 20 finnish news sites

9d39fe2759747c2cdbb2e6135c082c069e3d4a61 authored over 7 years ago by Lagittaja <[email protected]>
main script: fix accessing keys for upload to Internet Archive. Bump version to 20161122.01.

bbc3d4ea557bc89498479e88a39f84c4d02051d5 authored about 8 years ago by Arkiver2 <[email protected]>
main script: fix deduplication not working. Bump version to 20161121.01.

0d3bf1a152ec559c528dd882f3138c26e6071cd4 authored about 8 years ago by Arkiver2 <[email protected]>
Add two Iranian newssites.

d465ffa4d42cf2f83bce2fd22bec4a6622ae457d authored about 8 years ago by Arkiver2 <[email protected]>
Merge pull request #89 from readerboy7/patch-1

adding nzherald to list

314880ec8405292b6500d54c8da64b4292be38d9 authored about 8 years ago by Arkiver2 <[email protected]>
adding nzherald to list

a7efe3890296d46f2e10c15f67f647b1d7ff7dbb authored about 8 years ago by readerboy7 <[email protected]>
main script: use backup of targets.json if backup is found and targets.json is not found.

9e0ebc0a9cf1f794dcefcaea09965ab3a78e4c57 authored about 8 years ago by Arkiver2 <[email protected]>
Grab scripts: Fix type. Bump version to 20161114.01.

dba57f2e240b3a1447cc43392824f8152c25475b authored about 8 years ago by Arkiver2 <[email protected]>
Fix concurrent uploads not changed by IRC command for grabber. Bump version to 20161107.02.

6f42408cfdbb872e5f0dee097b6d33e48600a8f1 authored about 8 years ago by Arkiver2 <[email protected]>
Update for main scripts. Script to add keys.

Various fixes in uploading and IRC.
Bump version to 20161107.01.

07eb4a91f6848f11c95464ff58d95e027b5abb5f authored about 8 years ago by Arkiver2 <[email protected]>
Fix IRC string problem in discoverer. Bump version to 20161107.01.

cde969eee4861e7916145922ecaadfefdeadddaf authored about 8 years ago by Arkiver2 <[email protected]>
Fix upload function problem for grabber. Bump version to 20161107.01.

db8e131bc9f4f3b7173c69be0569ef5c11954faf authored about 8 years ago by Arkiver2 <[email protected]>
Fix imports for main server. Bump version to 20161106.01.

f2303d3fa2a430fea20a3ccb4b08e4a252a495a1 authored about 8 years ago by Arkiver2 <[email protected]>
Update services version to 20161106.01.

aa540186c3a11d05db199824b070e81869ab8a2e authored about 8 years ago by Arkiver2 <[email protected]>
Change refresh setting in services to seconds.

3deba7a3323fcc0df91073fd33d64e3d840fe257 authored about 8 years ago by Arkiver2 <[email protected]>
Fix pausing and resuming. Add upper limit for upload concurrent. Fix URL regex provblem. Bump version to 20161010.01.

86354fb769b2710ac7cdd5bd8795e43158b70ad2 authored about 8 years ago by Arkiver2 <[email protected]>
Main: fix if -> while. Wait for IRC to connect. Bump version to 20161003.01.

0dcae146a095f7ce99d41880eea3d1660a705cb8 authored over 8 years ago by Arkiver2 <[email protected]>
Discovery: wait for IRC bot to connect. Bump version to 20161003.01.

421d51722cc08f25848712afa1550b4fe9d9888b authored over 8 years ago by Arkiver2 <[email protected]>
Grab: fixes if -> while. Wait for the IRC bot to connect. Bump version to 20161003.01.

2ea01352e7833e87c864c00a1e37ab2d39df4780 authored over 8 years ago by Arkiver2 <[email protected]>
Fix variable with self. Bump main server version to 20161002.06.

a1a9f5fa0ec6cfb7625a1c271560bc60523eba99 authored over 8 years ago by Arkiver2 <[email protected]>
Fix KeyError on is_are. Bump main server version to 20161002.05.

c18723267b34ea833b3ce5457ec3bc5d09d37cb8 authored over 8 years ago by Arkiver2 <[email protected]>
Fix IRC reporting to 15 minutes for main server. Bump main server version to 20161002.03.

6a3f84ec18fe99abf1a71b6922a7a86455b6c282 authored over 8 years ago by Arkiver2 <[email protected]>
Import psutil for main server. Bump version to 20161002.03.

bf0663e189059355a6c3e0fe01d96dfe6861d8c2 authored over 8 years ago by Arkiver2 <[email protected]>
Fix !server-stats for all servers. Bump version to 20161002.02 for al servers.

742c82935dd5268ec2e9356d8ce328dd07319b0d authored over 8 years ago by Arkiver2 <[email protected]>
Remove empty targets.json file

088f53711eb14354ba25ed4a96e7b0ae2bf7f3d9 authored over 8 years ago by Arkiver2 <[email protected]>
Update IRC channels and nicks for the servers.

1a245fa086807592f96b8d2974930daf6eadaf97 authored over 8 years ago by Arkiver2 <[email protected]>
remove cubanet service

9fa4bced8b1b356dcb41fbbdd245fb666238c916 authored over 8 years ago by Arkiver2 <[email protected]>
Update new scripts to version 20161002.01.

Move old script to directory 'old', keep worker_script.py in main
directory for now.

794afd555df51cabaa115e3fb73a9a5e9f7c976a authored over 8 years ago by Arkiver2 <[email protected]>
README.md for main server scripts.

b8295863730d9824daa349270d39b78af7139d0d authored over 8 years ago by Arkiver2 <[email protected]>
Add psutil install and !server-stats IRC command to discovery server README.

a2eccd3c27f585cc9f7c26c095684980f0ce1275 authored over 8 years ago by Arkiver2 <[email protected]>
Add immediate-grab IRC command.

c1899372bbdc108cb181c94a3819f5271e3b60d9 authored over 8 years ago by Arkiver2 <[email protected]>
Discovery server README.md

05735a3682f8e8098ed6a0a33adf1b39ebd58e48 authored over 8 years ago by Arkiver2 <[email protected]>
add guccifer2.wordpress.com

5f4443e579190f3a0b26d54611dc05717e89b695 authored over 8 years ago by Start <[email protected]>
Merge pull request #76 from JesseWeinstein/wikidata_links

And even more wikidata links

77fb9274bae2ba1623511d22e2e7f261773dfeb2 authored over 8 years ago by HarryC145 <[email protected]>
Fix typo

228665007d5742f772573122ba07a904c153f9cd authored over 8 years ago by Jesse Weinstein <[email protected]>
Merge pull request #83 from lucasRolff/master

Add danish news sites

a8f754713b283d78ae98e8b4b98d75d16d76e174 authored over 8 years ago by HarryC145 <[email protected]>
Add danish news sites

Add some danish news sites

df97f4b02a1819a9825ed4754216f11fe4f57d50 authored over 8 years ago by Lucas Rolff <[email protected]>
Improve IRC commands.

Add good pause possibilities.
Upload of WARCs to Internet Archive improved.
Minor fixes.

52b39761894416e6a73c751d0588533941b8bbfa authored over 8 years ago by Arkiver2 <[email protected]>
Initial discovery server scripts.

0c6b7056f525ae88cb53a865f1ae6a7deb040133 authored over 8 years ago by Arkiver2 <[email protected]>
Initial grab server scripts.

03ea9f73b7511d543c47ddfbc114b1c97a6ea504 authored over 8 years ago by Arkiver2 <[email protected]>
Bump version to 20160718.01.

Add upload support of WARCs using internetarchive.
Add read_json and write_json to file.File.
Ad...

23c89c7da59abee733299ffe37ede70c78d0bdbf authored over 8 years ago by Arkiver2 <[email protected]>
Initial main server script.

9e685a452812d16ecb925b6bcd97ac6284eef643 authored over 8 years ago by Arkiver2 <[email protected]>
Merge pull request #82 from ArchiveTeam/HarryC145-patch-1

Update worker_script.py

2c9cda053f828e0c8c807e99e8e56cead8ed37f1 authored over 8 years ago by Arkiver2 <[email protected]>
Update worker_script.py

add the check on the ready folder

ff99c64a9e5d412a986e9529e85427be8ab2edc1 authored over 8 years ago by HarryC145 <[email protected]>
worker_script.py: don't process files in 'old_lists' and 'new_lists' dirs.

55e1cfe5b025f3edb0b24e702227ab87376b246f authored over 8 years ago by Arkiver2 <[email protected]>
Create rsyncd.conf

e9a5ca8f47652e1425a52f7efde77d4c8ad66b6e authored over 8 years ago by HarryC145 <[email protected]>
web__notinet_icrt_cu.py: fix

b618883c9b96c5ddcf990b100ec40277af83418e authored over 8 years ago by Arkiver2 <[email protected]>
18 Cuban sites/youtube channels.

451d12adc7635cea337eef081400447832a4ee20 authored over 8 years ago by Arkiver2 <[email protected]>
Add 3 blogs/newssites.

9e9a32395055d0be92733b8ed9fad591972542ac authored over 8 years ago by Arkiver2 <[email protected]>
Fix crash when loading old URL lists.

Bump version to 20160624.03.

c0ff738f8070f0dd2826c0b235a86f99894bd26d authored over 8 years ago by Arkiver2 <[email protected]>
Fix crash while loading bad URL.

Bump version to 20160624.02.

8766c9a80043afd3cb8eb07a7d2cd3bbf40c3183 authored over 8 years ago by Arkiver2 <[email protected]>
Change default IRC server to 'irc.servercentral.net'.

Check if new upload are allowed and/or max_concurrent_upload is reached
before creating new thre...

e1cdba962eb288c2aad13c9ba80393ed96a97875 authored over 8 years ago by Arkiver2 <[email protected]>
Add 6 Dutch business sites.

60ec8b4545c78b6ab1bff2cfdd2902172a8a6381 authored over 8 years ago by Arkiver2 <[email protected]>
Add main site.

4c24563e33d5264914cb5498aeb1840a099d2ed2 authored over 8 years ago by Arkiver2 <[email protected]>
Use utf-8.

Make case insensitive.
Bump version to 20160622.01.

c2a6b5f3165bef29290ad22da55ab58252a4ec58 authored over 8 years ago by Arkiver2 <[email protected]>
Create web__feweek_co_uk.py

4bcdf2348f362b657d7b7e3ef1c0cf2945fc4549 authored over 8 years ago by HarryC145 <[email protected]>
Create web__fullfact_org.py

045bf6bb824bcc5eed39a6b3c4f07dc934fa3cd1 authored over 8 years ago by HarryC145 <[email protected]>
Update web__lesoir_be.py

89d19d8692812c82df0e4c59fa1fbc315ff71cf0 authored over 8 years ago by HarryC145 <[email protected]>
Update web__destandaard_be.py

a3f70cca284a4fa0a2a889d05f44f766aba1be58 authored over 8 years ago by HarryC145 <[email protected]>
Update web__destandaard_be.py

3ae17bbdd6c774a6c4c91d955ef9365b4fec819f authored over 8 years ago by HarryC145 <[email protected]>
Update web__demorgen_be.py

f7ffe9e9d7b54fd44af50eb9f9e08412ed9aad75 authored over 8 years ago by HarryC145 <[email protected]>
Merge pull request #79 from sollidius/master

BE sites fixed

baba46922568f3502c2a16dfa35919acbc58216d authored over 8 years ago by HarryC145 <[email protected]>
Update web__lesoir_be.py

fixed

b464e56d430be942d76943330e73972be579c7f1 authored over 8 years ago by sollidius <[email protected]>
Create web__lesoir_be.py

144d7f5e3f87d0f752dac8bb96e18978dd800405 authored over 8 years ago by sollidius <[email protected]>
Create web__destandaard_be.py

ab2ea425cc2e2c8b7e999fed4fb1ce836967badc authored over 8 years ago by sollidius <[email protected]>
Create web__demorgen_be.py

49ec4a8017defd19c43bfffc130ce0cdcc45f857 authored over 8 years ago by sollidius <[email protected]>
Create web__gva_be.py

94094330f5ce30a330afa1229af35b3b11f68724 authored over 8 years ago by sollidius <[email protected]>
Update web__hln_be.py

805420b5a0b0a9389b99c608170bb902747cfd79 authored over 8 years ago by sollidius <[email protected]>
HLN.be

d17d7e51de51fb3c912a2dd0f6de0c8c064d0013 authored over 8 years ago by sollidius <[email protected]>
worker_script.py: do not requeue lists to prevent loops.

b81386f185d0ffd8b11ce47c8ef52da259a57ef6 authored over 8 years ago by Arkiver2 <[email protected]>
worker_script.py: accept grab-site exit code 256.

10df4da4402a5b70b8d4d75fe6f7d602acb88e36 authored over 8 years ago by Arkiver2 <[email protected]>
4 more wikidata links

5d6cc595b76526c6ccbe493f3b8c93d769c08e04 authored over 8 years ago by Jesse Weinstein <[email protected]>
worker_script.py: fix name too long issue.

aa629dff60101ec25603e3d93044e702279a6b52 authored over 8 years ago by Arkiver2 <[email protected]>
worker_script.py: debugging for listname.

37b6aa9d7958ffbb25bb0e1cfc9baf67ac6f8104 authored over 8 years ago by Arkiver2 <[email protected]>
worker_script.py: fix name too long issue.

2657f52e2ffc7d996b7bd3ad45f868689a103058 authored over 8 years ago by Arkiver2 <[email protected]>
worker_script.py: add debug line to get returned_code from grab-site.

14a6bbdcccdc6ea67a9b8ad518ebece7bb866e5c authored over 8 years ago by Arkiver2 <[email protected]>
Add grabscript execution to own thread.

Check if an URL is an URL, to prevent crashes.
Remove exceptions printing of problematic URLs.
B...

b7cb950e1e0cbaca9c260a89b637407d641092cd authored over 8 years ago by Arkiver2 <[email protected]>
Create web__irna_ir.py

45a409e9181a9b0cd37de353ce59da442fb34946 authored over 8 years ago by HarryC145 <[email protected]>
Create web__thedailystar_net.py

3091424dbae4bef097e1b065ebcc80ed2c47dfdd authored over 8 years ago by HarryC145 <[email protected]>
Create web__tehrantimes_com.py

b5d5208f163953681621c799fc64999b0249c2eb authored over 8 years ago by HarryC145 <[email protected]>
Update web__leparisien_fr.py

de98493f3b82bdd85e6ef58bfb816ac4df9face7 authored over 8 years ago by HarryC145 <[email protected]>
Update web__leparisien_fr.py

11d98943ca9346eaebef74a9b24f5d4b3c84aa0e authored over 8 years ago by HarryC145 <[email protected]>
Create web__leparisien_fr.py

4f9ba03e2b364d05e3b0453968dd7eda0698c904 authored over 8 years ago by HarryC145 <[email protected]>
Create web__dprktoday_com.py

0b10f81a36ad45b4050f06fb82d2bcaa1f67e161 authored over 8 years ago by HarryC145 <[email protected]>
Fix size-hint.

Bump to version 20160404.01.

10c35536f740fbd64a1f6d970e6568e9480cc88b authored almost 9 years ago by Arkiver2 <[email protected]>
worker_script.py: retry grab on a list of URLs if grab did not return status code 0.

ab4a33156a175171915057b5bde46f588d14e3ec authored almost 9 years ago by Arkiver2 <[email protected]>
Add domain as first URL in 'urls'.

e9f741f37d3509baee18d93a2ab627fd80794292 authored almost 9 years ago by Arkiver2 <[email protected]>
Add discoverer script.

Fix PING IRC problem.
Fix writing URL lists.
Remove discovery lines.
Add support for discoverer ...

cd6b84041dc6ecf4aec4d9768f5b100110f03f11 authored almost 9 years ago by Arkiver2 <[email protected]>
26 Argentina newssites and YouTube channels.

cbf099553139cf20f881828e598766bc58be4560 authored almost 9 years ago by Arkiver2 <[email protected]>
Antigua and Barbuda newssites.

c648ae2a29a3b52c725cc8b064f75fb19d4e9cdd authored almost 9 years ago by Arkiver2 <[email protected]>
11 Angola newssites and YouTube channels.

16f20434a75a78ab2113b58b996da9b338132f53 authored almost 9 years ago by Arkiver2 <[email protected]>
Fix web__economictimes_indiatimes_com.

e9ac66ecfcac0d30270b73b07883ac5a745dd5f4 authored almost 9 years ago by Arkiver2 <[email protected]>
Merge pull request #71 from BnA-Robin/master

Added missing services from the Alexa Top 40 News Sites

222829ca8856873a2975d08187605e827dcdc40f authored almost 9 years ago by Arkiver2 <[email protected]>