Ecosyste.ms: OpenCollective

An open API service for software projects hosted on Open Collective.

github.com/ArchiveTeam/IA.BAK

We back up a lot of stuff from around the web; now it's time to back up the Internet Archive, just in case.
https://github.com/ArchiveTeam/IA.BAK

finished shard5; activate shard6

a420adbe0be5a23c2eac1e32f6f9587c954f3c1e authored over 9 years ago by Joey Hess <[email protected]>
fall back to dumb temp file when dumb os lacks tempfile

b8b09df2d2698987a68ca9667ce624d38bfa06d7 authored over 9 years ago by Joey Hess <[email protected]>
ANNEXGETOPTS file can be used to enable concurrent downloads (-J10)

b0691e9a704109ea50c2765886a4c3be2b89b9bf authored over 9 years ago by Joey Hess <[email protected]>
when iabak is interrupted, or exits, kill all iabak-hourlysync processes (which now write their pid to a file)

c2b322617d6cc4a26a0fb8096e3f968d1c87b441 authored over 9 years ago by Joey Hess <[email protected]>
add shard7 reserve (8.8 tb)

a64f5fab4600c9b4ef3a77b78c000eb606cd623f authored over 9 years ago by Joey Hess <[email protected]>
Merge branch 'master' of github.com:ArchiveTeam/IA.BAK

f99508050eae45e765700aa1d63b8a456b217060 authored over 9 years ago by Joey Hess <[email protected]>
shard4 is done; set to maint and activate shard5

76896ae7c389798910ec8d92bdb21d46c01ff193 authored over 9 years ago by Joey Hess <[email protected]>
Merge branch 'remove-reminder'

e850f5768a625f792b0046a7b8ecc895ba841dfd authored over 9 years ago by Daniel Brooks <[email protected]>
retry syncs if they fail

Tries up to 5 times, and has an exponential backoff to avoid
congestion (which is the most commo...

63f685368f1ba74d09d9d980e19faae03d3183cd authored over 9 years ago by Daniel Brooks <[email protected]>
remove the cronjob reminder, and always make sure the cronjob is set up

0add1e764a6769b4581fb6b7653095c19cc9d4b5 authored over 9 years ago by Daniel Brooks <[email protected]>
a few wording tweaks

78adea8cbf4b6d0d635265a1f087a5f2cdad113f authored over 9 years ago by Joey Hess <[email protected]>
multiple doc updates

f5e680a655b28903b17c961728082d58901df2b3 authored over 9 years ago by Joey Hess <[email protected]>
Merge pull request #21 from db48x/cronjob

automatically install a systemd service/cron job

bc9f3c73a227896edc3febe0bbb81293d5f7a8cc authored over 9 years ago by Joey Hess <[email protected]>
add a message about allowing the current user to linger

42fc2231b1e26a152f9eafcfd7dda25bc8452133 authored over 9 years ago by Daniel Brooks <[email protected]>
Merge branch 'master' into cronjob

9332cbbfbbdbb914bd13872f4479912b37b6aaf2 authored over 9 years ago by Daniel Brooks <[email protected]>
set shard3 to maint (108 files not downloaded due to a url glitch but we have enough clients)

94eed16b74a19189bf96d76a515df93eba9eb6bb authored over 9 years ago by Joey Hess <[email protected]>
use AccuracySec=6h rather than picking 2:02:02AM by hand

This will make systemd pick a random time between midnight and
6am (local time) to run the job. ...

5b8924ae046ac5e359856aadcf8feaecebb40d56 authored over 9 years ago by Daniel Brooks <[email protected]>
shard5 and shard6 now exist, in reserve

c560bb1679df1af1a561f78cfef8acd66d352fbb authored over 9 years ago by Joey Hess <[email protected]>
shard3 is filling; set shard4 to active as well

95beefa925829439aa38f23c8b4e51bc1cf2b396 authored over 9 years ago by Joey Hess <[email protected]>
detect if dirname -z is not supported and use alternate code path

ce320386c94bd1ba0a48645cde911dee127bc061 authored over 9 years ago by Joey Hess <[email protected]>
automatically install a systemd service/cron job

This happens during setup; run ./install-fsck-service to do it later.

845bc1749775ad13ee49227a25d45ab7e2c37870 authored over 9 years ago by Daniel Brooks <[email protected]>
hello shard4

1e147bc45913f2ead605ce0ce5c1efb7a459dd51 authored over 9 years ago by Joey Hess <[email protected]>
document NOMORE

53b4bfc50a8dcd695ba0f4309ceea033834b8b39 authored over 9 years ago by Joey Hess <[email protected]>
move clients on to new shards when they finish, as long as they still have disk space

Also, if an active shard is not available, pick a reserve shard.

6a38a741f46d5b17738f7fad427c706e5977680f authored over 9 years ago by Joey Hess <[email protected]>
temporarily call change-email so that old shards get the email registered

2bd78fbbf4c6fdee9d54a00d69d8936af6986919 authored over 9 years ago by Joey Hess <[email protected]>
shard2 is in maint mode

a712e8cb67723446a18fc2921d9d6d709123bc87 authored over 9 years ago by Joey Hess <[email protected]>
typo

0362cd94192de5038472c61a68cf3562bf2534f7 authored over 9 years ago by Joey Hess <[email protected]>
weird, CGI.pm doesn't escape + when generating urls

c7c0cade77f7d6d65c5cfdbe3fe03e97ef3e9583 authored over 9 years ago by Joey Hess <[email protected]>
stupid fqdn check

38e2966f5df62866dcfcf676055ca50dda81bd09 authored over 9 years ago by Joey Hess <[email protected]>
tiny amount of email address checking

b994f1a77027124add42458dc76de4d47b93f9ff authored over 9 years ago by Joey Hess <[email protected]>
use CGI.pm to generate urls

2d9a171ac24c53542d44158b81236b90b0de0b3b authored over 9 years ago by Joey Hess <[email protected]>
typo

26d71575129ac2d1087820c6664382b871fa7fe2 authored over 9 years ago by Joey Hess <[email protected]>
add uuid to registration process

25bcf0639db684a05bffb10dd718a598fda30b96 authored over 9 years ago by Joey Hess <[email protected]>
support for changing registration email

69fdb86b9bdd0abd3745cb030ea5c0a659a0a425 authored over 9 years ago by Joey Hess <[email protected]>
refactor

22bf7beb0d321277a9af7c33e1efbde75d776275 authored over 9 years ago by Joey Hess <[email protected]>
bugfix

c49a398eda643e1cddfe81792e966b87ca0c3c50 authored over 9 years ago by Joey Hess <[email protected]>
deal with + in pubkey

70df7d9d22d501cd04d65ac6ab3f61ef2526d908 authored over 9 years ago by Joey Hess <[email protected]>
fixes

62996fee3aa2ae92ab6ba097eb935235ae88cdf1 authored over 9 years ago by Joey Hess <[email protected]>
inverted

ab3031fc759187ffacd3dd9a78d9981934a6d3cf authored over 9 years ago by Joey Hess <[email protected]>
add key

7948465658cdf2d78a2a133eef0a153b0374e254 authored over 9 years ago by Joey Hess <[email protected]>
prompt for registration email, and use cgi script to register

f09ff080b91fc748baca257630d09e0e61522e6e authored over 9 years ago by Joey Hess <[email protected]>
OSX ready, it seems

41b06057ede680e1bb1788cc8e5f221597e0cd05 authored over 9 years ago by Joey Hess <[email protected]>
sleep 3600 because 1h is too much to expect a command to parse on a user-friendly system like OSX

1b804a75f57bb1e5387e9a72e144a7d4b60bc66f authored over 9 years ago by Joey Hess <[email protected]>
use osx path

47ca8270a8d4d442377ea3433cee41c57948537a authored over 9 years ago by Joey Hess <[email protected]>
avoid find -printf for OSX port

635b1b13d3ba001bd6162fb1d3ca9cc1b630d344 authored over 9 years ago by Joey Hess <[email protected]>
use distributionVersion, not distributionReleasedate

They are not always the same. Also, this is much simpler.

aa9bfb52a64b78844fc31e525d585b1fde05f636 authored over 9 years ago by Joey Hess <[email protected]>
more fun

c66ad7a30dad39c3054ae25dc414941fa220e545 authored over 9 years ago by Joey Hess <[email protected]>
remove temp dir

c8bf488ed3b0e6c9b98a270932b2334208344e43 authored over 9 years ago by Joey Hess <[email protected]>
remove major version

452ac3de9963c657502bdfd001b7223aaa8443f7 authored over 9 years ago by Joey Hess <[email protected]>
fix fix

5839ebb3c8340d01be80880a15c3d413e314a637 authored over 9 years ago by Joey Hess <[email protected]>
fix

2afc0ac9c016b7a882911d4bbefc3bfce6ed958b authored over 9 years ago by Joey Hess <[email protected]>
use git-annex version --raw because OSX's sed is crap

feac202e83da42877c88fac2cfacdcfa9ee9a591 authored over 9 years ago by Joey Hess <[email protected]>
shut up perl warnings

a26666b00898177ceecaf3e8891dd0d97834e125 authored over 9 years ago by Joey Hess <[email protected]>
typo

f671a2babeb54c08bf7f5086d086c003d29851f3 authored over 9 years ago by Joey Hess <[email protected]>
use perl instead of find for OSX port

e090a7d590c283d30b687d4f22cd4269fd338673 authored over 9 years ago by Joey Hess <[email protected]>
port git-annex installation code to work on OSX

e6bd83b42e8f1d519ddf44fcda031466558dbf77 authored over 9 years ago by Joey Hess <[email protected]>
no longer need a prerelease of git-annex; today's 420 release will do

97c27e2918d0c0f89204251a9528e2a81033e91e authored over 9 years ago by Joey Hess <[email protected]>
avoid readlink -f which is not portable to OSX

9339421e9708735eb10c9123d5fed7b27087749f authored over 9 years ago by Joey Hess <[email protected]>
print reminder if cron job has not been run

5001ccb58fd1ea5d1a39acea8a98d7fd816aace6 authored over 9 years ago by Joey Hess <[email protected]>
added iabak-cronjob and support FSCKTIMELIMIT config file

4a021baa3edcecf7dac34e10037e89a5b7440219 authored over 9 years ago by Joey Hess <[email protected]>
activate shard3

e4c63c09fd8fae9bc95dd85dd9a050fccd945f3e authored over 9 years ago by Joey Hess <[email protected]>
switch fast fask to be a truely fast expiry-preventing ping

1dc6c53578949c922116decb24c6af417f323da6 authored over 9 years ago by Joey Hess <[email protected]>
automatically nice and ionice self where available

3abcfbbdc0510e78e4fde0db099d8f569fbfba56 authored over 9 years ago by Joey Hess <[email protected]>
temporarily switch to the git-annex autobuild, to get activity.log bugfix

2f0c8ab40bac2048391a48f299057f6cb31f1b7b authored over 9 years ago by Joey Hess <[email protected]>
use FLOCK for fscks

a62ea081794b14e30831b42f7372e30405bbc109 authored over 9 years ago by Joey Hess <[email protected]>
update the .gitignore file

6e2784449bc70259807cdc0c15da0ef3406f924c authored over 9 years ago by Daniel Brooks <[email protected]>
Merge branch 'master' of github.com:ArchiveTeam/IA.BAK

c1f04b2fe7894f1c758d6f5ba0e682dd1e44a95b authored over 9 years ago by Daniel Brooks <[email protected]>
provide an alternative to flock(1) written in perl

It's not a complete replacement, but we only use it one way so this
doesn't matter. Only used if...

a32c37f556cd7df6f1ac35b54005ce77caa96cf8 authored over 9 years ago by Daniel Brooks <[email protected]>
Merge pull request #13 from cancerAlot/master

bugfix when there is no internet connection

21331b4d4c4cf3203b042bd0fa834a3ea2563d66 authored over 9 years ago by Daniel Brooks <[email protected]>
Update install-git-annex

c4f46abed7fe0cb7559fd71605222b5c2d6ff79f authored over 9 years ago by cancerAlot <[email protected]>
fixes a simple bug when there is no internet connection

(installedVersion is "" and -lt says NaN)

e84796df5f2a24b1e3d4ce0f702756b825cedb79 authored over 9 years ago by cancerAlot <[email protected]>
give example in form numfmt can parse (git-annex can parse either form)

3e78ff90207c5434c5925f632060c1591352524b authored over 9 years ago by Joey Hess <[email protected]>
make fast fsck use --in here, which is faster

This avoids checking location log etc for files that git-annex thinks are
not in the local repo....

8728a04fa80f492d314cb2db90e50d0d1345686a authored over 9 years ago by Joey Hess <[email protected]>
use lock file to prevent more than one fsck at a time when run concurrently

c898e5a2023fc0a92c7f0842966d2320a8c2a2a8 authored over 9 years ago by Joey Hess <[email protected]>
tell user shard name

14ea0b446a7b15ee8324d86bcf81e13ee509e15d authored over 9 years ago by Joey Hess <[email protected]>
assign new users at random to an active shard

c57bf560d37967a412fe1b24dd093b4000f1b305 authored over 9 years ago by Joey Hess <[email protected]>
+x

160356fcc7aa1bfae55dad5276f2cb72f02c7916 authored over 9 years ago by Joey Hess <[email protected]>
lock when installing git-annex to prevent concurrency problems

This is a blocking lock.

71a7c4b94e8576ebb6d4fecc3b1a8a20dee5613d authored over 9 years ago by Joey Hess <[email protected]>
move some config details from the wiki

c0e06b112ee2e466c1d24574f81cac2f6a3466ca authored over 9 years ago by Joey Hess <[email protected]>
start work on shard2 immediately after checkout

478134d43144e9e0f0c5e33244488ff080efbf4c authored over 9 years ago by Joey Hess <[email protected]>
improve message

0f1d13d84dd6988c3a79e46e70132727292c652e authored over 9 years ago by Joey Hess <[email protected]>
skip already checked out shard

cf3ed9eee1c79859e0bdcd564ced958f49f1d053 authored over 9 years ago by Joey Hess <[email protected]>
add NOMORE control file

7bdc606c88581f29b17f1d347fd0b2d4e6de6344 authored over 9 years ago by Joey Hess <[email protected]>
get all clients to checkout shard2, unless they've touched NOMORE

fb202cb9e331bbf659d846b9d7d3184caccf4cd8 authored over 9 years ago by Joey Hess <[email protected]>
avoid redundant whole-directory downloads for 1 missing file in maint mode

13b07a4b714a49963dd7fe216eedd3498b5d8d99 authored over 9 years ago by Joey Hess <[email protected]>
simplif

824fee8416c4b314d886e5a8846174d16f75942c authored over 9 years ago by Joey Hess <[email protected]>
more haskellitus

bb01f4abe0b73c5edc00c56f0819c283fd858c99 authored over 9 years ago by Joey Hess <[email protected]>
avoid ./ in find output

69cd7b69e085c3c868bbd71093b78691f99ffa70 authored over 9 years ago by Joey Hess <[email protected]>
no fall-thru

783f35eacc2c5ad8d80680befcffef9e89c1d082 authored over 9 years ago by Joey Hess <[email protected]>
thought I was writing haskell for a sec

0e416a6b0149ad01ea77acd2fbc071b914d3feb8 authored over 9 years ago by Joey Hess <[email protected]>
remove unused funct stub

a0fdc55041634fa7d193f2d23456df02eaec30cc authored over 9 years ago by Joey Hess <[email protected]>
begin handling different states differently

Also, sync before starting download, otherwise an out of date shard repo
could result in unncess...

32165aa1a63a5dfa51474b11adf1452ace768481 authored over 9 years ago by Joey Hess <[email protected]>
SHARD1 done! activate SHARD2

5a15958d97d714e7d755b052bd33c8a590781a24 authored over 9 years ago by Joey Hess <[email protected]>
fix url to force immediate key deploy

8c57f7ad67b19b821e280f02280654ffbe2cd625 authored over 9 years ago by Joey Hess <[email protected]>
hit webhook to pull new keys in registration process (github's web hook seems slow or not always arriving)

b1346211a2cf939ebf8e333f45d5dacb1d8329b8 authored over 9 years ago by Joey Hess <[email protected]>
docs

c35feeb9580ed1054fb47098c42aae428afcfa2a authored over 9 years ago by Joey Hess <[email protected]>
find prevshard before we create a new shard

ab2c2c9882c190c4358b9ca6326f42e3ea3cf5b0 authored over 9 years ago by Joey Hess <[email protected]>
sort by ctime

0d431596b12aae6806cd7b67efc51fd5897acd23 authored over 9 years ago by Joey Hess <[email protected]>
better, just grep -i for shard names

1ca901e0f30c203102cade385a26621c20992c88 authored over 9 years ago by Joey Hess <[email protected]>
accept uppercase SHARD names

e67be5b9184bebd9e77e75c18c59c3dec571a832 authored over 9 years ago by Joey Hess <[email protected]>