Ecosyste.ms: OpenCollective
An open API service for software projects hosted on Open Collective.
github.com/ArchiveTeam/NewsGrabber-Main
The scripts for the main NewsGrabber server.
https://github.com/ArchiveTeam/NewsGrabber-Main
true-up from master
fdd77c832de8b2c8de544c836ed3325fca8911c5 authored about 6 years ago by root <[email protected]>
fdd77c832de8b2c8de544c836ed3325fca8911c5 authored about 6 years ago by root <[email protected]>
Support warrior files.
Bump version to 20170617.01.
8db4f730059dbb64b32c17fc80bdd304998814ec authored over 7 years ago by Arkiver2 <[email protected]>
Skip bad JSON files. Bump version to 20170101.01.
9eb5c232f3dbd784bef3cb06bc232f07a85050a3 authored almost 8 years ago by Arkiver2 <[email protected]>
9eb5c232f3dbd784bef3cb06bc232f07a85050a3 authored almost 8 years ago by Arkiver2 <[email protected]>
Fix typo. Bump version to 20161205.01.
0092b18b9760f39d2d2ac9751dc12e3a97629b15 authored about 8 years ago by Arkiver2 <[email protected]>
0092b18b9760f39d2d2ac9751dc12e3a97629b15 authored about 8 years ago by Arkiver2 <[email protected]>
Do not deduplicate WARCs larger than 1 GB to prevent OOM.
Bump version to 20161203.01.
d700723191f8e07b16f04ddb6dac6cf0cb440071 authored about 8 years ago by Arkiver2 <[email protected]>
Temporary skip of problematic WARCs during deduplication.
Bump version to 20161125.06.
b49a68a89c97371c7b2a0115f48fb6dcc5496e39 authored about 8 years ago by Arkiver2 <[email protected]>
Fix removing *.upload file after deduplication.
Bump version to 20161125.05.
9e80779b250f6e33d18ae4005ea91227bc957f85 authored about 8 years ago by Arkiver2 <[email protected]>
Deduplicate WARCs one by one.
Bump version to 20161125.04.
3d71e04f38ef0d1762101e192367bc5e614ca7d0 authored about 8 years ago by Arkiver2 <[email protected]>
Fix file creation before name is created.
Bump version to 20161125.03.
3ff6c89fb1494b7a1562123782c508bdcdf0d1fb authored about 8 years ago by Arkiver2 <[email protected]>
Prevent of uploading of WARCs while being deduplicated.
Bump version to 20161125.02.
008ecc0c4df8d3318466738654fdcea1b6643ae9 authored about 8 years ago by Arkiver2 <[email protected]>
Deduplicate WARCs before uploading.
Bump version to 20161125.01.
c1115ef7d48aaff88560a08f5e420285cc0fdfea authored about 8 years ago by Arkiver2 <[email protected]>
fix for starting script for the first time
52a2414c0a7d7551a27943955ece8ca346aac630 authored about 8 years ago by Arkiver2 <[email protected]>
52a2414c0a7d7551a27943955ece8ca346aac630 authored about 8 years ago by Arkiver2 <[email protected]>
initial
ebed6050652c841cfa561a333adf86a0deff6757 authored about 8 years ago by Arkiver2 <[email protected]>
ebed6050652c841cfa561a333adf86a0deff6757 authored about 8 years ago by Arkiver2 <[email protected]>