Archive Team commits | Ecosyste.ms: OpenCollective

Add error handling

github.com/ArchiveTeam/Ubuntu-Warrior - 833a607a8ef14c20270fb2f9433f222e90ca4f2e authored about 1 year ago by tech234a <[email protected]>

Minor comment tweaks

github.com/ArchiveTeam/Ubuntu-Warrior - 865108802447da9b7d2460f77681ebf33128122c authored about 1 year ago by tech234a <[email protected]>

Comment fixes

github.com/ArchiveTeam/Ubuntu-Warrior - eb24adb621c3ae775d84fd761fc1bb1e803b4341 authored about 1 year ago by tech234a <[email protected]>

Enforce the weekly reboot on the error message

github.com/ArchiveTeam/Ubuntu-Warrior - c05583e43fbd3d55bc31f7fe590da871e5f23180 authored about 1 year ago by tech234a <[email protected]>

Add a sleep on the error message

github.com/ArchiveTeam/Ubuntu-Warrior - 48ad9ed57fb24b337941530beffea7a378d2a1ea authored about 1 year ago by tech234a <[email protected]>

Drop support for 3.0, 3.1, 3.2-beta

github.com/ArchiveTeam/Ubuntu-Warrior - cdf218f76bb7d1b8c948c551b0798b4a1b7118ed authored about 1 year ago by tech234a <[email protected]>

Attempt fix for 3.1

github.com/ArchiveTeam/Ubuntu-Warrior - f7ea182f3820c6dd1f85cc43b5c976dae16785ae authored about 1 year ago by tech234a <[email protected]>

Revert an attempted fix for 3.0

github.com/ArchiveTeam/Ubuntu-Warrior - 0f14f2a8e8d84626879ce871989d1275bbc58bb1 authored about 1 year ago by tech234a <[email protected]>

Update startup.sh

github.com/ArchiveTeam/Ubuntu-Warrior - a54095f4c4f816d8279eb93ffccea3110170cdeb authored about 1 year ago by tech234a <[email protected]>

Fix Warrior 3.0

github.com/ArchiveTeam/Ubuntu-Warrior - 7147ff6a07868e8c0914b072f160f42ec1c50713 authored about 1 year ago by tech234a <[email protected]>

Prevent errors while waiting for the reboot

github.com/ArchiveTeam/Ubuntu-Warrior - 75c339003e3d93aeb963db2408b509d4841499a0 authored about 1 year ago by tech234a <[email protected]>

Update startup.sh

github.com/ArchiveTeam/Ubuntu-Warrior - 7ba36e49a5332f03865d347fffc93e1735797bde authored about 1 year ago by tech234a <[email protected]>

Perform Alpine/Docker updates as needed

github.com/ArchiveTeam/Ubuntu-Warrior - dcc3cd442331f8fbcdcdfa87cf7107b2b166fb3a authored about 1 year ago by tech234a <[email protected]>

replace "yield from" with "await" in tests

github.com/ArchiveTeam/ludios_wpull - 7e0e32378f8edf0f5230ea5b19fb48531edc472e authored about 1 year ago by HeliosLHC <[email protected]>

Merge branch 'python-3.11' of https://github.com/ArchiveTeam/ludios_wpull into python-3.11

github.com/ArchiveTeam/ludios_wpull - 7f81a38e3841a1238d7414ce32b6d0e9ef7df05e authored about 1 year ago by HeliosLHC <[email protected]>

imports cleanup

github.com/ArchiveTeam/ludios_wpull - 0d75a5acb58b2559cc2eda9282c1104c4d0bced9 authored about 1 year ago by Nyx <[email protected]>

cleanup unused imports

github.com/ArchiveTeam/ludios_wpull - 78e8fdd5b7abe195c2ebea7114e5feea7be19150 authored about 1 year ago by Nyx <[email protected]>

remove unnecessary await

github.com/ArchiveTeam/ludios_wpull - 19a0640ec0a1d54a070ca2bfc50e2a2e61d91baf authored about 1 year ago by Nyx <[email protected]>

Change LinkContext to frozen dataclass to enable hashing

github.com/ArchiveTeam/ludios_wpull - f63acfc567ee057cec6d9235a6aa05154351ee8c authored about 1 year ago by Nyx <[email protected]>

fix incorrect typing for "link_type"

github.com/ArchiveTeam/ludios_wpull - 7f1e6ef3c88ee765ffcb9d54e24234b23ba8bb05 authored about 1 year ago by Nyx <[email protected]>

move LinkInfo and convert to dataclass

github.com/ArchiveTeam/ludios_wpull - f0f59807c4f9f5049153e1ec9199430173a9d359 authored about 1 year ago by Nyx <[email protected]>

Version 20231213.01. New Wget-AT.

github.com/ArchiveTeam/warrior-dockerfile - 86f9433661b11031c30a0031ea561140728c21c8 authored about 1 year ago by arkiver <[email protected]>

Version 20231213.01. New Wget-AT.

github.com/ArchiveTeam/grab-base-df - ef4072fe98c4b4f8cde7b480a57af3a85df86bd5 authored about 1 year ago by arkiver <[email protected]>

Version 1.21.3-at.20231213.03. Write Wget build information to warcinfo record. Add Archive Team contact details to --version and --help messages.

github.com/ArchiveTeam/wget-lua - 28bec41b7f31352115077999696b3f8ea873a02e authored about 1 year ago by arkiver <[email protected]>

Upgrade to Python 3.11 asyncio syntax

github.com/ArchiveTeam/ludios_wpull - 9ef03cf2bbff4fec4bba417d4066c2cfa2a16a6f authored about 1 year ago by Nyx <[email protected]>

Merge pull request #24 from HeliosLHC/modernizations

replaced namedlist and ordereddefaultdict with stdlib implementations

github.com/ArchiveTeam/ludios_wpull - 17bc682cff16428585b39ed118006d61658c1ca1 authored about 1 year ago by HeliosLHC <[email protected]>

autopep8 reformatting

github.com/ArchiveTeam/ludios_wpull - a42a5154a40200164fa33428ac16d351a71175ce authored about 1 year ago by HeliosLHC <[email protected]>

ignore venv directory

github.com/ArchiveTeam/ludios_wpull - f884cd1740a546a78af6e87c90762948d01575ee authored about 1 year ago by HeliosLHC <[email protected]>

replace namedtuples with dataclasses

github.com/ArchiveTeam/ludios_wpull - ab1f8a9aacc0092a05d39999f5ee6e85ed6fae06 authored about 1 year ago by HeliosLHC <[email protected]>

remove unused imports

github.com/ArchiveTeam/ludios_wpull - 25076b5192616d0c7a203f4c480db4dbf185697a authored about 1 year ago by HeliosLHC <[email protected]>

Version 1.21.3-at.20231213.02. Use HTTP/1.1 for the request, not HTTP/1.0.

github.com/ArchiveTeam/wget-lua - 0e7d903275ee835970c8343dcf5e66a65c9168cd authored about 1 year ago by arkiver <[email protected]>

Version 1.21.3-at.20231213.01. Write WARC-Protocol and WARC-Cipher-Suite WARC headers. Fix writing HTTP URL to WARC headers when transformed to HTTPS with HSTS. Fix setting specific single SSL/TLS protocol version instead of minimum version.

github.com/ArchiveTeam/wget-lua - d7855b0a875a87fbe90f9757064f7c28dae38c0d authored about 1 year ago by arkiver <[email protected]>

Version 20231212.01. Exit URL with ver URL parameter.

github.com/ArchiveTeam/urls-grab - a730f5492d690b5f8d30fb00be9f394273f7aad3 authored about 1 year ago by arkiver <[email protected]>

Version 20231212.01. Build new image to use Debian bookworm.

github.com/ArchiveTeam/warrior-dockerfile - 28a0ab6af71e02411c81cecd397e94191095921f authored about 1 year ago by arkiver <[email protected]>

Use Debian bookworm.

github.com/ArchiveTeam/grab-base-df - 1ba1d4d6a57b1032e9dd67bd21fdde02c342ada0 authored about 1 year ago by arkiver <[email protected]>

Use Debian bookworm.

github.com/ArchiveTeam/wget-lua - c9820674ffe7c68751f77fb690a6d3ec765d268d authored about 1 year ago by arkiver <[email protected]>

Version 20231207.01. Exit the URL on finding the xoid URL parameter.

github.com/ArchiveTeam/urls-grab - 1bfe2ffb6e5e52c8dfe23ca555e5dcc7d5d88fb6 authored about 1 year ago by arkiver <[email protected]>

swap namedtuple with dataclass

github.com/ArchiveTeam/ludios_wpull - eee42fd2d5f89ab2fca6f052b9608cf34e125492 authored about 1 year ago by HeliosLHC <[email protected]>

Merge pull request #566 from JustAnotherArchivist/dashboard-websocket-reduce-load

Reduce server load from the WebSocket server

github.com/ArchiveTeam/ArchiveBot - ad9703c489168bb88cd53b3f4ea3b6dadfe8820f authored about 1 year ago by JustAnotherArchivist <[email protected]>

Reduce server load from the WebSocket server

Previously, the server was encoding and compressing each message for every client. This resulted...

github.com/ArchiveTeam/ArchiveBot - 03b33766c5833769fb08fd5e7a5545607e5811b2 authored about 1 year ago by JustAnotherArchivist <[email protected]>

Version 20231201.01. Change protocol.

github.com/ArchiveTeam/reddit-grab - 48dc016fafb69d032c056819a74d319b4806bb67 authored about 1 year ago by arkiver <[email protected]>

Version 20231129.04. Support dynamicviews blogs.

github.com/ArchiveTeam/blogger-grab - d738bcfba7a5bdda578ae0d6e8710c5573716d8c authored about 1 year ago by arkiver <[email protected]>

Merge pull request #565 from JustAnotherArchivist/ssl-openssl3-unsafe-renegotiation

Reenable unsafe renegotiation (cf. RFC 5746) and update openssl.conf for OpenSSL 3.x

github.com/ArchiveTeam/ArchiveBot - dd5ce62dc52eaf3307f8109723fbda192f72613a authored about 1 year ago by JustAnotherArchivist <[email protected]>

Version 20231129.03. Do not abort on adult blogs anymore.

github.com/ArchiveTeam/blogger-grab - fc39f94d93e12d17a7a25c5b0effb37ea6d8d9da authored about 1 year ago by arkiver <[email protected]>

Version 20231129.02. Detect loop in path and ignore URLs at loop depth 4.

github.com/ArchiveTeam/blogger-grab - 7bc7d6f98afdb911241e30888b6654903acfcb74 authored about 1 year ago by arkiver <[email protected]>

Version 20231129.01. Do not skip extracting items from gallery if it would not be deleted (this is now a continuous project).

github.com/ArchiveTeam/imgur-grab - efbcb5d3e2883f2aebc2b167d1ea216c27248212 authored about 1 year ago by arkiver <[email protected]>

Version 20231129.01. Add resolv.conf file.

github.com/ArchiveTeam/blogger-grab - 358db83b9397632a54fe376c4610dfa82ce42230 authored about 1 year ago by arkiver <[email protected]>

Reenable unsafe renegotiation (cf. RFC 5746) and update openssl.conf for OpenSSL 3.x

github.com/ArchiveTeam/ArchiveBot - 26eea1af479201e8815336766596014aa4bf22e5 authored about 1 year ago by JustAnotherArchivist <[email protected]>

Version 20231127.02. New --ciphers value.

github.com/ArchiveTeam/reddit-grab - 5f7cee8d3a482b09809c677d397f4f79a714cc15 authored about 1 year ago by arkiver <[email protected]>

Version 20231127.02. Add clock and invalid domain tests.

github.com/ArchiveTeam/blogger-grab - 14f2eddfdc2b6d78dbd3f5c282a6f607f0029426 authored about 1 year ago by arkiver <[email protected]>

Version 20231127.01. Use --ciphers SECURE256.

github.com/ArchiveTeam/reddit-grab - 2b41d8ef429438d02ac9988110167c7ad5d9072e authored about 1 year ago by arkiver <[email protected]>

Version 20231127.01. Do not get /search/label/ URLs containing ?.

github.com/ArchiveTeam/blogger-grab - 97f217f9964a81dcd814dd7e7178785827649b76 authored about 1 year ago by arkiver <[email protected]>

Version 20231126.03. Disable processing of Wget-AT extracted URL due to problems filtering out sidebars and footers.

github.com/ArchiveTeam/blogger-grab - 0eb9827cf68c33086210640cb4ac50d609942a64 authored about 1 year ago by arkiver <[email protected]>

Version 20231126.02. Enable default Wget-AT URL extraction.

github.com/ArchiveTeam/blogger-grab - 4fc9549335390a730d634407c46d6a091312c7b7 authored about 1 year ago by arkiver <[email protected]>

Version 20231126.01. Relax pattern to extract URLs from href=.

github.com/ArchiveTeam/blogger-grab - 2c554b7dd02e8d8a61496d6b8922cd8d1df9b4fb authored about 1 year ago by arkiver <[email protected]>

Merge pull request #12 from bogsen/patch-1

Fix slow pattern for identifying JS script versions

github.com/ArchiveTeam/telegram-grab - 816235ee72dc3b1b889fdc2f936ae8b0fbf1e344 authored about 1 year ago by arkiver <[email protected]>

Version 20231125.01. Do not accept /search URL with too many slashes.

github.com/ArchiveTeam/blogger-grab - 87b7053c36eb55bdf4891ef0b4e644a3cf7984ac authored about 1 year ago by arkiver <[email protected]>

Version 20231124.02. Ignore searchsearchsearch label URL.

github.com/ArchiveTeam/blogger-grab - e22fa1768e9f4b5e81948810a7b4215678aea3f1 authored about 1 year ago by arkiver <[email protected]>

Version 20231124.01. Ignore ?en-xx URLs. Handle /search/?q= URLs.

github.com/ArchiveTeam/blogger-grab - 7edfd2aefaa78f8ef4547c778e801e39ca723e18 authored about 1 year ago by arkiver <[email protected]>

Version 20231123.04. Ignore blogger.com/feeds/*/blogs URLs.

github.com/ArchiveTeam/blogger-grab - ae954e013a98666f397bff7aaa64faaa4ecd465e authored about 1 year ago by arkiver <[email protected]>

Version 20231123.03. Only retry on problematic blog:* item URL.

github.com/ArchiveTeam/blogger-grab - 9ca9b8a2b40d716a9e5d7b2f8cfec2b17f66a37b authored about 1 year ago by arkiver <[email protected]>

Version 20231123.02. Timeout of 4 seconds. Remove --no-http-keep-alive option.

github.com/ArchiveTeam/blogger-grab - 3b14a3fde0293dad6d8a1529607006a5cc607a55 authored about 1 year ago by arkiver <[email protected]>

Version 20231123.01. Do not do retries on url:* item.

github.com/ArchiveTeam/blogger-grab - c06375d9859cde09ddcdfb1c8178421a12753750 authored about 1 year ago by arkiver <[email protected]>

Version 20231121.19. Fix support for non-com TLDs for blogspot and blogger when extracting blog from URL.

github.com/ArchiveTeam/blogger-grab - 1f1735a9f0efe37fde8219b06ab2701321e7a101 authored about 1 year ago by arkiver <[email protected]>

Version 20231121.18. Support all TLDs. Prevent odd /search/label/ loop. Take out .htmlfeeds/posts URLs.

github.com/ArchiveTeam/blogger-grab - 58d22bd566415973b2de0456037eaa83685c5c3e authored about 1 year ago by arkiver <[email protected]>

Version 20231121.17. Ignore share-post.g? URL.

github.com/ArchiveTeam/blogger-grab - 66832a5851dfe148eebc0fe958eabcb50809abd7 authored about 1 year ago by arkiver <[email protected]>

Version 20231121.16. Do not print message on finding beginning and end of footer/sidebar/menu.

github.com/ArchiveTeam/blogger-grab - 734922b3a80de2929c68946658b1f3c1eb7db986 authored about 1 year ago by arkiver <[email protected]>

Version 20231121.15. Ignore search?q= URLs without updates-max parameter. Ignore ?m%3D1?m%3D1 URLs.

github.com/ArchiveTeam/blogger-grab - ca7be6b96bdf5427eca04b8ed031bdd9d3c544c1 authored about 1 year ago by arkiver <[email protected]>

Version 20231121.14. Ignore URLs with more than 5 slashes or more.

github.com/ArchiveTeam/blogger-grab - 95455f66f64aebbafce506732fec788c41bfe6e8 authored about 1 year ago by arkiver <[email protected]>

Version 20231121.13. Support discovering blog, article, and page items using old *.blogger.com domain.

github.com/ArchiveTeam/blogger-grab - 0e154edf1064e8b082d0a0754e07a428a6b1d91a authored about 1 year ago by arkiver <[email protected]>

Version 20231121.12. Remove debug output.

github.com/ArchiveTeam/blogger-grab - 300f089a327b5c074f61a5ea90268fd3f4a47aa9 authored about 1 year ago by arkiver <[email protected]>

Version 20231121.11. Fix handling https:///.

github.com/ArchiveTeam/blogger-grab - 108aa9b0ea3e9e2a6911c7f24d2dcd623fff621f authored about 1 year ago by arkiver <[email protected]>

Version 20231121.10. Do not extract items from menu, sidebar, and footer for article, page, and search items.

github.com/ArchiveTeam/blogger-grab - 4cbaf15783c56b6a83f464f672c854def76e03a7 authored about 1 year ago by arkiver <[email protected]>

Version 20231121.09. Ignore ?m=0 URLs.

github.com/ArchiveTeam/blogger-grab - f1e23fef7b9236d0de160ea9d6736015676826a4 authored about 1 year ago by arkiver <[email protected]>

Version 20231121.08. Get followed.g list of followers.

github.com/ArchiveTeam/blogger-grab - 60bbaba2c5be79ba1f13fd24ac6ff5f8a2cc1c81 authored about 1 year ago by arkiver <[email protected]>

Version 20231121.07. Do not print every discovered item. Multi item size 1000 (limit on tracker side).

github.com/ArchiveTeam/blogger-grab - 4cbf738a9ce8e7f34447c588cea18f6a0e038960 authored about 1 year ago by arkiver <[email protected]>

Version 20231121.06. Support profile items.

github.com/ArchiveTeam/blogger-grab - 584c4578c6fabc0b28cad15495d5f021b6a5f50a authored about 1 year ago by arkiver <[email protected]>

Fix slow pattern for identifying JS script versions

github.com/ArchiveTeam/telegram-grab - 7a3404cbbbc9218799a62ed51ef3ac6f224e11bb authored about 1 year ago by Bogdan Seniuc <[email protected]>

Version 20231121.05. Write outlinks to a special stash project.

github.com/ArchiveTeam/blogger-grab - 61460697e7f8bef30a1325685f356f5caad0ca57 authored about 1 year ago by arkiver <[email protected]>

Version 20231121.04. Take out sitemap.xml check in robots.txt.

github.com/ArchiveTeam/blogger-grab - 2047dd5ebd3a294efad5764ecec2086263654c1a authored about 1 year ago by arkiver <[email protected]>

Version 20231121.03. Keep decoding items before matching for finding aborted items.

github.com/ArchiveTeam/blogger-grab - 9ee77725d8dbc986450583cb911f9071bd580413 authored about 1 year ago by arkiver <[email protected]>

Version 20231121.02. Move item name check.

github.com/ArchiveTeam/blogger-grab - 8f88ad54dd7b639ec4c6f7ed0f695ff6a3e1546e authored about 1 year ago by arkiver <[email protected]>

Version 20231121.01. Fix local where needed.

github.com/ArchiveTeam/blogger-grab - 2c768d1c819058bfbbd1f1711512bed80fdea4b3 authored about 1 year ago by arkiver <[email protected]>

Version 20231120.02. Do 2 retries on a problematic URL. Ignore ?widgetType=BlogArchive URLs.

github.com/ArchiveTeam/blogger-grab - 0193896f4df3cebe99d9e3699a16e0e1f845545b authored about 1 year ago by arkiver <[email protected]>

Version 20231120.01. Ignore ?showComment= URLs.

github.com/ArchiveTeam/blogger-grab - a8e86a8ed26fccc3d2e6aa597e834a145f0d26be authored about 1 year ago by arkiver <[email protected]>

Version 20231119.09. Increase backfeed size to 500 items.

github.com/ArchiveTeam/blogger-grab - 57081be4557d578fa7a11d6bf5ed600c86acbcf3 authored about 1 year ago by arkiver <[email protected]>

Version 20231119.08. Skip URLs with space characters.

github.com/ArchiveTeam/blogger-grab - 5549daf465e9977407db5e7ae69a4b846a8c6d62 authored about 1 year ago by arkiver <[email protected]>

Version 20231119.07. Multi item size 200 to limit a tracker side. Relax sitemap check.

github.com/ArchiveTeam/blogger-grab - 89476a7d22ff3db747bad498a605dbc88ae89c0f authored about 1 year ago by arkiver <[email protected]>

Version 20231119.06. Various updates.

github.com/ArchiveTeam/blogger-grab - 17a130b7a17279177a9b4ba3d8ea0b62b3617d89 authored about 1 year ago by arkiver <[email protected]>

Version 20231119.05. Get update-max pages again.

github.com/ArchiveTeam/blogger-grab - 3d78441dcb12136502dbdd902bacee30bd39a2ea authored about 1 year ago by arkiver <[email protected]>

Version 20231119.04. Multi item size 10. Correctly use cjson. Accept status code 400. Ignore main pagination due to size. Better ignores.

github.com/ArchiveTeam/blogger-grab - 672e4a6cc2b2f006f710e7688f1defa97ed44724 authored about 1 year ago by arkiver <[email protected]>

Version 20231119.03. Fix project name.

github.com/ArchiveTeam/blogger-grab - 359e24b509475c601081b65445d278272a18dc89 authored about 1 year ago by arkiver <[email protected]>

Version 20231119.02. Move checks for skipping a blog up. Skip on adult overlay.

github.com/ArchiveTeam/blogger-grab - de5683e117c4d0e10c7d80c604cc013a176a053d authored about 1 year ago by arkiver <[email protected]>

Version 20231119.01. Initial.

github.com/ArchiveTeam/blogger-grab - e473f3c84b0b51f481c377a192ed03e3b5bb8d10 authored about 1 year ago by arkiver <[email protected]>

Version 20231119.01. Reuse connections.

github.com/ArchiveTeam/telegram-grab - ff4276e3815a2cd5ee6812a3871d6176b9875878 authored about 1 year ago by arkiver <[email protected]>

Version 20231118.01. Switch to gnutls.

github.com/ArchiveTeam/reddit-grab - 7da27ab11036a75bb514378e3b62fc4c4488251f authored about 1 year ago by arkiver <[email protected]>

Version 20231115.01. Change cipher list.

github.com/ArchiveTeam/reddit-grab - 0dc36e31e004f8f31981500c0918c535a3eb6ae7 authored about 1 year ago by arkiver <[email protected]>

Version 20231112.04. Stop queuing URLs discovered on special interest pages.

github.com/ArchiveTeam/urls-grab - 394d1a56af634518ee05e7fbedde578c636e3b33 authored about 1 year ago by arkiver <[email protected]>

Version 20231112.03. Stop queuing to the Zippyshare project.

github.com/ArchiveTeam/urls-grab - f6b1d712035d4d431cbecd28807c0c9b7aed50bf authored about 1 year ago by arkiver <[email protected]>

Version 20231112.02. Discover and queue Imgur items.

github.com/ArchiveTeam/urls-grab - 5a6d47040d20dd1aa181f5299815b81479dc3640 authored about 1 year ago by arkiver <[email protected]>