Archive Team commits | Ecosyste.ms: OpenCollective

Version 20230727.01. Support latest Wget-AT.

github.com/ArchiveTeam/telegram-grab - eb43a7472822d8c34dec46b5f35d420e7dec2007 authored over 1 year ago by arkiver <[email protected]>

Version 20230725.02. Enable queuing URLs from special interest URLs.

github.com/ArchiveTeam/urls-grab - 047a74b99276328953523d31d0b6efbae11e6299 authored over 1 year ago by arkiver <[email protected]>

Version 20230725.01. Reformat checks in pipeline.

github.com/ArchiveTeam/urls-grab - c77610711a4878da35f31030484808e5f61347af authored over 1 year ago by arkiver <[email protected]>

Version 20230722.03.

github.com/ArchiveTeam/urls-grab - ff80c50c254e96955179edaed8521af5d6ca952a authored over 1 year ago by arkiver <[email protected]>

Revert "Version 20230722.02. Queue pages from special interest pages again."

This reverts commit 2d78071caa26421348929e5526277eb0246baabf.

github.com/ArchiveTeam/urls-grab - 85fd95763f744b669689b56df213234d71616160 authored over 1 year ago by arkiver <[email protected]>

Bump version 4.0

github.com/ArchiveTeam/warrior4-vm - 1677f2e2ef927281dc212bc99d204fc454f714cd authored over 1 year ago by Christopher Foo <[email protected]>

Version 20230722.02. Queue pages from special interest pages again.

github.com/ArchiveTeam/urls-grab - 2d78071caa26421348929e5526277eb0246baabf authored over 1 year ago by arkiver <[email protected]>

Version 20230722.01. Extract more news sitemaps.

github.com/ArchiveTeam/urls-grab - 5d7bd3f1ebc5bfa72adf48d355572a082da13887 authored over 1 year ago by arkiver <[email protected]>

Version 20230721.01. Fix pattern match problem.

github.com/ArchiveTeam/urls-grab - 735b4fde33595843a078681913bd788ff12176d4 authored over 1 year ago by arkiver <[email protected]>

Version 20230719.03. Improve spam check.

github.com/ArchiveTeam/urls-grab - 3c6d283ba5a06aafa447c45d220cf446430fc27b authored over 1 year ago by arkiver <[email protected]>

Version 20230719.02. Queue default pages only for https URL.

github.com/ArchiveTeam/urls-grab - 280dbb1d95d39442db1c32e89b606ac9e446f116 authored over 1 year ago by arkiver <[email protected]>

Version 20230719.01. Only queue special URLs if main page is 200.

github.com/ArchiveTeam/urls-grab - 859f85668310d2f15b92503cf6bd66ea03dfe779 authored over 1 year ago by arkiver <[email protected]>

Merge pull request #19 from OrIdow6/traceback-with-lua-errors

Print traceback with Lua errors.

github.com/ArchiveTeam/wget-lua - d7530c88e745beaff2622f2e676d20c5f82fd49e authored over 1 year ago by arkiver <[email protected]>

Version 20230718.01. Extract changed decoding function.

github.com/ArchiveTeam/youtube-grab - 5b89a483ff63d136914266012b1664c4a76bbab1 authored over 1 year ago by arkiver <[email protected]>

Version 20230718.02. Improve spam loop checks.

github.com/ArchiveTeam/urls-grab - 74dd2276d440f60c05edb08d405f4707e9ae9bcd authored over 1 year ago by arkiver <[email protected]>

Version 20230718.01. Improve spam loop checks.

github.com/ArchiveTeam/urls-grab - a76de59a4b711f741844a3585f3285252252aeea authored over 1 year ago by arkiver <[email protected]>

Merge pull request #560 from ivan/log-window-scroll-down

github.com/ArchiveTeam/ArchiveBot - b22627e4cd94784d970a2555e071840536af71e1 authored over 1 year ago by Ivan Kozik <[email protected]>

dashboard: fix out-of-viewport log windows not being scrolled down in Chrome

github.com/ArchiveTeam/ArchiveBot - bb39bf8e4a78f1f8ac4c6ca49ee69aac9dc1307a authored over 1 year ago by Ivan Kozik <[email protected]>

Revert "dashboard: remove `content-visibility` CSS to fix log windows not being scrolled to the bottom after expanding them in Chrome"

This reverts commit c99d488875468bb00fec190f26df1658359eed07.

This regressed performance in Chr...

github.com/ArchiveTeam/ArchiveBot - 7ad41d8e4bd22cd833d62e1f49e28fe4f4c3237d authored over 1 year ago by Ivan Kozik <[email protected]>

dashboard: remove `content-visibility` CSS to fix log windows not being scrolled to the bottom after expanding them in Chrome

github.com/ArchiveTeam/ArchiveBot - c99d488875468bb00fec190f26df1658359eed07 authored over 1 year ago by Ivan Kozik <[email protected]>

Merge pull request #559 from ivan/dashboard-query-args

github.com/ArchiveTeam/ArchiveBot - efacb6d78aebed90803e967d8e8e6d64da676c86 authored over 1 year ago by Ivan Kozik <[email protected]>

dashboard: use `URLSearchParams` instead of our own query arg parser

that did not decode URL escapes like %5E, making it impossible to put
a ^ in the filter.

github.com/ArchiveTeam/ArchiveBot - ecd91c05b10c1e32b266ba20bb20b438124a0a86 authored over 1 year ago by Ivan Kozik <[email protected]>

dashboard: remove our `intoObject` and use `Object.fromEntries`

github.com/ArchiveTeam/ArchiveBot - a30ed2be144e89325f6f0759604b02ecf55a6875 authored over 1 year ago by Ivan Kozik <[email protected]>

Mention Open Collective page.

github.com/ArchiveTeam/standalone-readme-template - 626d8b167aff35e5fdf5778311a2793c1ab12a7d authored over 1 year ago by arkiver <[email protected]>

Version 20230716.01. Change spam loop check.

github.com/ArchiveTeam/urls-grab - a5689410b86fe3814157ffdfbccd108c6fcf7eac authored over 1 year ago by arkiver <[email protected]>

Print traceback with Lua errors.

github.com/ArchiveTeam/wget-lua - 1903fca838bfac84c14a23809ba6fd02fa142df7 authored over 1 year ago by OrIdow6 <[email protected]>

Version 20230711.02. Do not extract URLs from bcyimg items.

github.com/ArchiveTeam/banciyuan-grab - b16562e5265d62650fd44c52338886aa4c09e354 authored over 1 year ago by arkiver <[email protected]>

Version 20230711.01. Switch to banciyuan-images tracker and bcyimg item support.

github.com/ArchiveTeam/banciyuan-grab - 747f482292a483139448a5b898f4332a62ef65ba authored over 1 year ago by arkiver <[email protected]>

Version 20230711.02. Do not check 'res' in http_stat when deciding to write to WARC.

github.com/ArchiveTeam/urls-grab - f77b4c4a4af1354e94f90b7a57cd2953e7acec67 authored over 1 year ago by arkiver <[email protected]>

Version 20230711.01. Queue news sitemaps to separate project.

github.com/ArchiveTeam/urls-grab - 83c2b3d7312f096e98166c9f73aead5b7583cbb9 authored over 1 year ago by arkiver <[email protected]>

Version 20230708.03. Check for correctly extracted URLs from robots.txt.

github.com/ArchiveTeam/urls-grab - 067de6151f9a8bca3e188b7157e44aafb3969008 authored over 1 year ago by arkiver <[email protected]>

Version 20230708.02. New attempt at prevent spam loop.

github.com/ArchiveTeam/urls-grab - 5a9f3981d7130a6817bac0c909b2b2bafebbf633 authored over 1 year ago by arkiver <[email protected]>

Version 20230708.01. Relax checks on spam domain URLs.

github.com/ArchiveTeam/urls-grab - ca89daa9a67e47f7bd08326414af9717ea6cbcec authored over 1 year ago by arkiver <[email protected]>

Version 20230707.03. Attempt to fix spam loop detection.

github.com/ArchiveTeam/urls-grab - 70685f77585462be87b5c2ae1c2431ab26746976 authored over 1 year ago by arkiver <[email protected]>

Version 20230707.02. Attempt to block out several spam loops.

github.com/ArchiveTeam/urls-grab - 7c67e83f626f7e18306f452c189a86af3d138e2c authored over 1 year ago by arkiver <[email protected]>

Version 20230707.01. Exit URLs with various parameters. Exit URLs with path starting with various default paths.

github.com/ArchiveTeam/urls-grab - 37b31234ec574567fb12046c1fb310ea7e7da3f2 authored over 1 year ago by arkiver <[email protected]>

Version 20230706.02. Move around exit URL check.

github.com/ArchiveTeam/urls-grab - 840dc650011b8c5f6320d0bf726b01c0b6124f16 authored over 1 year ago by arkiver <[email protected]>

Version 20230706.01. Exit URL on state_uuid parameter.

github.com/ArchiveTeam/urls-grab - 890852df27c28428393c22b26fa7c124c613b1da authored over 1 year ago by arkiver <[email protected]>

Version 20230704.15. Reduce retries to 5.

github.com/ArchiveTeam/banciyuan-grab - 76699e6e6ab7f0399ee2bd05d31813d0836aabc7 authored over 1 year ago by arkiver <[email protected]>

Version 20230704.14. Make sure page parameter is moved to back for all occurences of main item: URL.

github.com/ArchiveTeam/banciyuan-grab - a7d9a8f350f0841471e64ae87dfd3763b25e0a65 authored over 1 year ago by arkiver <[email protected]>

Version 20230704.13. Move page parameter of initial item: URL to back.

github.com/ArchiveTeam/banciyuan-grab - 2e93bcd0f54dc3466a28460bd9649ade3f1c1046 authored over 1 year ago by arkiver <[email protected]>

Version 20230704.12. Two checks on if item likely does not exist.

github.com/ArchiveTeam/banciyuan-grab - a2905f148c35613cd1b346afa862f7628e1fe050 authored over 1 year ago by arkiver <[email protected]>

Add initial patching support

github.com/ArchiveTeam/warrior4-vm - 2a592dfde5bc67dd3a3c5cd9ebc0f56dbdc99f64 authored over 1 year ago by Christopher Foo <[email protected]>

Version 20230704.11. Skip profile /like page. Retry on bad SSR data.

github.com/ArchiveTeam/banciyuan-grab - e7be31f18503dc076d1c21b74da9f79d4845dd11 authored over 1 year ago by arkiver <[email protected]>

Version 20230704.10. Only queue reply API URLs if there are replies.

github.com/ArchiveTeam/banciyuan-grab - a57cb1987a01062bd921b4caf59e61a949763869 authored over 1 year ago by arkiver <[email protected]>

Version 20230704.09. Do not queue the follow API URL without page parameter.

github.com/ArchiveTeam/banciyuan-grab - bf4f75a2fd4ecf9711109767643eea7b4595f484 authored over 1 year ago by arkiver <[email protected]>

Version 20230704.08. Fewer requests. Earlier status code check.

github.com/ArchiveTeam/banciyuan-grab - 16f788b3834ba88251d3999e3a6574dc6b5c4006 authored over 1 year ago by arkiver <[email protected]>

Version 20230704.07. Set max tries to 7.

github.com/ArchiveTeam/banciyuan-grab - 288325a0441b172bb9a75b24abda9b03ddb61ee4 authored over 1 year ago by arkiver <[email protected]>

Version 20230704.06.

github.com/ArchiveTeam/banciyuan-grab - 4bab4e429694b91564e57f6ee5998c6bce944b38 authored over 1 year ago by arkiver <[email protected]>

Version 20230704.05. Take out ?filter= page.

github.com/ArchiveTeam/banciyuan-grab - 41eea4c5cda04b691f34eac4c6e5895b986331f5 authored over 1 year ago by arkiver <[email protected]>

Version 20230704.04. Prevent some data being queued twice.

github.com/ArchiveTeam/banciyuan-grab - 08ac3ae1add29fc9f4cbc6e15037597a6dc0a492 authored over 1 year ago by arkiver <[email protected]>

items from JSONL file

github.com/ArchiveTeam/banciyuan-items - 41a67a1c63ae230d76c2d42410f0638a2dbbb1f4 authored over 1 year ago by arkiver <[email protected]>

Version 20230704.03. Fix fix_case. Disable fix_case.

github.com/ArchiveTeam/banciyuan-grab - 79f848f4d1caf29ac398289089cf961600bf25ff authored over 1 year ago by arkiver <[email protected]>

Version 20230704.02. Backfeed keys.

github.com/ArchiveTeam/banciyuan-grab - bf004ccbb5ba2e3bb3d7872896fd782eece6ffea authored over 1 year ago by arkiver <[email protected]>

Version 20230704.01. Initial.

github.com/ArchiveTeam/banciyuan-grab - 84cbda0bb70a238009a3eae272766355afe7033c authored over 1 year ago by arkiver <[email protected]>

Bump version 4.0-beta.3

github.com/ArchiveTeam/warrior4-vm - 0cd3cb2d691621e559d6238a4cde2f4f9ea82d49 authored over 1 year ago by Christopher Foo <[email protected]>

Set a hostname (#6)

github.com/ArchiveTeam/warrior4-vm - 8d1700bd32576b70ad6ed2182e2ea4cfc395bf14 authored over 1 year ago by Christopher Foo <[email protected]>

Version 20230630.04. Use random Extra header on cdnvideo.ru. Abort item faster on 404 on api.yarus.ru.

github.com/ArchiveTeam/yarus-grab - dbb02ccd765f505d0ea6ca014799328f29ca3bca authored over 1 year ago by arkiver <[email protected]>

Version 20230630.03. Higher retries again.

github.com/ArchiveTeam/yarus-grab - 8b0cfcb12ebf669d5d99e41ff97aba65277fe7e9 authored over 1 year ago by arkiver <[email protected]>

Version 20230630.02. Reduce max tries to 4.

github.com/ArchiveTeam/yarus-grab - d6bf2c21dab2df12dccc1edfbf4c1543dcd3b861 authored over 1 year ago by arkiver <[email protected]>

RAW files.

github.com/ArchiveTeam/yarus-items - f97f79098f71f7cf4191a695738b769de9da58fa authored over 1 year ago by arkiver <[email protected]>

Version 20230630.01. Initial.

github.com/ArchiveTeam/yarus-grab - 85e62c0f3abab513331a3f9763cf8c648b9fb6be authored over 1 year ago by arkiver <[email protected]>

items

github.com/ArchiveTeam/yarus-items - cb172806ffcb4a4db6daa9e63a5ac3141bc70c61 authored over 1 year ago by arkiver <[email protected]>

Merge pull request #558 from ivan/dashboard-modernization

Modernize the dashboard JavaScript and improve things

github.com/ArchiveTeam/ArchiveBot - 86e4009e849d9be81b94f15494ed1e241f16832f authored over 1 year ago by JustAnotherArchivist <[email protected]>

Tell user to wait when creating containers

github.com/ArchiveTeam/warrior4-vm - 1a2bb5fb105d43ba5abe6f2fefa41b38e351577e authored over 1 year ago by Christopher Foo <[email protected]>

Show eth0 IP address (#5)

github.com/ArchiveTeam/warrior4-vm - e4f4796404c839c87ae3e81d1527ef5e78679555 authored over 1 year ago by Christopher Foo <[email protected]>

items

github.com/ArchiveTeam/tiki-items - e1a8aa94cd6faba456b9ace1fe1c1780add27f79 authored over 1 year ago by arkiver <[email protected]>

Version 20230627.01. Randomize order of DNS servers for --dns-servers options.

github.com/ArchiveTeam/urls-grab - 2141d7593e31feb485747347823e3f828c311cde authored over 1 year ago by arkiver <[email protected]>

Merge pull request #14 from Hans5958/patch-2

Fix broken syntax, minor changes

github.com/ArchiveTeam/standalone-readme-template - 43364b92021bb0a354f37fb8354aff9fea6839dd authored over 1 year ago by arkiver <[email protected]>

Version 20230626.02. Write percent encoded URLs to aborted URLs files.

github.com/ArchiveTeam/urls-grab - efb4de13dad08486c2da2080b15eb56bb06d9de4 authored over 1 year ago by arkiver <[email protected]>

Version 20230627.02. Max tries to 9. Only get video pages if videos exist. Do not get followers information. Get video thumbnails on user items.

github.com/ArchiveTeam/tiki-grab - 24f6915cb28d2a9c2f20d04f13511b6cf2544efc authored over 1 year ago by arkiver <[email protected]>

Also adjust the other README

github.com/ArchiveTeam/standalone-readme-template - 509619870e0161be9e18a65e6c8e030228b5bf93 authored over 1 year ago by Hans5958 <[email protected]>

Fix broken syntax, minor changes

github.com/ArchiveTeam/standalone-readme-template - 1b9626080b02ab90b62e345a1eee12933c348efd authored over 1 year ago by Hans5958 <[email protected]>

Version 20230627.01. Queue outlinks directly to the urls project.

github.com/ArchiveTeam/reddit-grab - bb6198cc1a2a4f707ef7b7504aa4517c193e3427 authored over 1 year ago by arkiver <[email protected]>

Version 20230627.01. Initial.

github.com/ArchiveTeam/tiki-grab - 398c60b7b8620a8bc7d2e2bea757d67a317ccc66 authored over 1 year ago by arkiver <[email protected]>

User items

github.com/ArchiveTeam/tiki-items - 29d6f9cbb1beaeb739672a52f51fccc181f22eeb authored over 1 year ago by arkiver <[email protected]>

Merge pull request #13 from Hans5958/patch-1

Run images detached, some minor changes

github.com/ArchiveTeam/standalone-readme-template - f7647dafbaf3d5911ed3ed4bb0d804919b6ff3d2 authored over 1 year ago by arkiver <[email protected]>

Version 20230626.01. Do not queue all URLs found on special interest pages.

github.com/ArchiveTeam/urls-grab - 9a7b50188e1adbc863c0d9ee84dae44823e1a13a authored over 1 year ago by arkiver <[email protected]>

Run images detached, some minor changes

Based on https://wiki.archiveteam.org/index.php/Running_Archive_Team_Projects_with_Docker

github.com/ArchiveTeam/standalone-readme-template - 06259eb55ac8ae98ef0c7474f7ff3c3803eb06ab authored over 1 year ago by Hans5958 <[email protected]>

Version 20230625.03. Fix filters.

github.com/ArchiveTeam/urls-grab - 909200b97553cc6a08755b2bf12af65082f1c955 authored over 1 year ago by arkiver <[email protected]>

Version 20230625.02. Move filters from server to repo.

github.com/ArchiveTeam/urls-grab - bdcc9ce1f28072557bf0e86d966eba00362b7e4a authored over 1 year ago by arkiver <[email protected]>

Version 20230625.01. Exit on several parameters in URL.

github.com/ArchiveTeam/urls-grab - 352cc1ecb8e5377efe38f2c79373f25c377974cc authored over 1 year ago by arkiver <[email protected]>

3600_news_crypto_sites.txt

update url list

github.com/ArchiveTeam/urls-sources - 98865e3a97beace64f45f593ca2ea3171b8afb3c authored over 1 year ago by datechnoman <[email protected]>

Version 20230624.01. Get rid of another news/html spam loop.

github.com/ArchiveTeam/urls-grab - abc6f6f437c330e25873d2a77b67f968514cf674 authored over 1 year ago by arkiver <[email protected]>

Add files via upload

github.com/ArchiveTeam/urls-sources - 171d5d1d2b78fe78a82c80cc172c37c93c432723 authored over 1 year ago by datechnoman <[email protected]>

Version 1.21.3-at.20230623.01. Calculate WARC-Payload-Digest value over payload stripped of transfer encoding.

github.com/ArchiveTeam/wget-lua - 4fda5f2cc5188a1db15ff5b0d535d56aafc5a4de authored over 1 year ago by arkiver <[email protected]>

Version 1.21.3-at.20230621.01. Add options --reject-subnets SUBNETS, --accept-subnets SUBNETS, --reject-reserved-subnets.

github.com/ArchiveTeam/wget-lua - aa346a3a958692a2405b0cb35ccb2019e7dd772f authored over 1 year ago by arkiver <[email protected]>

Merge pull request #3 from reedy/patch-1

Update README.md

github.com/ArchiveTeam/warrior4-vm - 07df0ac5103bfb0aad8d03e7c01ba633ef834625 authored over 1 year ago by Christopher Foo <[email protected]>

Update README.md

* Documentation about OS to choose
* Document expanded disk size
* Document how to amend image...

github.com/ArchiveTeam/warrior4-vm - 0a4027c813e250ce444cc6f0febb42715ada4478 authored over 1 year ago by Sam Reed <[email protected]>

Fix compile warnings for luahooks_dedup_response.

github.com/ArchiveTeam/wget-lua - bda3c51ca5f3aec3b893a726b874b4f809ebba8e authored over 1 year ago by arkiver <[email protected]>

Version 20230619.02. Accept 404 on mediaembed URL.

github.com/ArchiveTeam/reddit-grab - f1ef7d169771c785c3182fccb1b04fbf1c3eb663 authored over 1 year ago by arkiver <[email protected]>

Version 20230619.01. Primitive fix to user post verification problems.

github.com/ArchiveTeam/reddit-grab - d2571cde06a5cb445b0427313d5734a59192a529 authored over 1 year ago by arkiver <[email protected]>

Version 20230617.01. Use --secure-protocol=auto for Wget-AT.

github.com/ArchiveTeam/reddit-grab - 2b19cdcd43ac0d252062cd33b142a0a7cdb0d4ea authored over 1 year ago by arkiver <[email protected]>

queued remaining post items.

github.com/ArchiveTeam/reddit-items - b5d72b4d5b635cf3bbb98f72409a7d7ab63ee49c authored over 1 year ago by arkiver <[email protected]>

added two post lists.

github.com/ArchiveTeam/reddit-items - 141691d44372a87d505f36b0f80aa5b232663fba authored over 1 year ago by arkiver <[email protected]>

Version 20230616.05. Use custom resolv.conf with 'search .'.

github.com/ArchiveTeam/urls-grab - 957b3bcb8747085085450ba096bee0f36ce1f195 authored over 1 year ago by arkiver <[email protected]>

Version 20230616.04. Move check to test.

github.com/ArchiveTeam/urls-grab - b30e76440d50d5d4a348ebbcc59e23cd558c303d authored over 1 year ago by arkiver <[email protected]>

Version 20230616.03. Relax max clock offset to 180 seconds.

github.com/ArchiveTeam/urls-grab - dacd2e1c9b99ea2b31c291e92aade3214bdd69b3 authored over 1 year ago by arkiver <[email protected]>

Version 20230616.02. Run checks every 30 multi items.

github.com/ArchiveTeam/urls-grab - f77165553638f0d1698aa3242c84e014622f3206 authored over 1 year ago by arkiver <[email protected]>

Version 20230616.01. Better debug output on connection checks.

github.com/ArchiveTeam/urls-grab - ce31538a46d3a9d7922f597c3240dd9f4b809edd authored over 1 year ago by arkiver <[email protected]>