Ecosyste.ms: OpenCollective
An open API service for software projects hosted on Open Collective.
github.com/ArchiveTeam/ArchiveBot
ArchiveBot, an IRC bot for archiving websites
https://github.com/ArchiveTeam/ArchiveBot
0dc4794bf67765c3b8339420b4fa672450f76d6f authored over 1 year ago by Ivan Kozik <[email protected]>
We don't really need a dynamic !con; it's easy enough to backspace and change the number yourself.
5508ac65eab208a4d82071949521610217376ccc authored over 1 year ago by Ivan Kozik <[email protected]>b1dadfbab1336543ae50c3dc0b19a50906265382 authored over 1 year ago by Ivan Kozik <[email protected]>
7373eddba29086ffcee62b7b0a83ba95c865476d authored over 1 year ago by Ivan Kozik <[email protected]>
35308c9471f959d2494c263790c3bd3c7a84dd6c authored over 1 year ago by Ivan Kozik <[email protected]>
7fc1752568a080059ba9cfc567f73fb243f6f835 authored over 1 year ago by Ivan Kozik <[email protected]>
4faa46b1bbf8f5f39eca05fbaa454e76af326ef4 authored over 1 year ago by Ivan Kozik <[email protected]>
e1d5905ba041cb8db435e91bb033e734ab8404a3 authored over 1 year ago by Ivan Kozik <[email protected]>
2bea325af4d0b79303dd42ee7cd48696985c7e04 authored over 1 year ago by Ivan Kozik <[email protected]>
Their performance wasn't good enough in Chrome, even on a 7950X3D.
cf8abc619548e975695f16a937d25f9d21e1a3e5 authored over 1 year ago by Ivan Kozik <[email protected]>The dashboard displays 2-3 times as many jobs as it used to, and we need to do something to miti...
c2cf7849e06f271bc626983436555d6c9a13f983 authored over 1 year ago by Ivan Kozik <[email protected]>6b62e8ff54e42d90635b3bc8d574254f0ffbe9c4 authored over 1 year ago by Ivan Kozik <[email protected]>
7946d7fa62ba3f2222a4cdfc9e51056210914260 authored over 1 year ago by Ivan Kozik <[email protected]>
7be1a8e46182e02a3669dcf1915a969bfa6b5cf8 authored over 1 year ago by Ivan Kozik <[email protected]>
227215457811a0a3aa93f79fe1e0a2ee916c0495 authored over 1 year ago by Ivan Kozik <[email protected]>
3d92ccd325619aaa6b43f6a3c78152ba1c760799 authored over 1 year ago by Ivan Kozik <[email protected]>
ab96deb4dfe628cd6f6891491629e9220459d0ae authored over 1 year ago by Ivan Kozik <[email protected]>
Documentation: change "www.bar.org" links to "example.net" links
7ad0d3882c7508d4190dc46fa5e66f1ae937e9ab authored almost 2 years ago by JustAnotherArchivist <[email protected]>7a42330b6b16d53db0d312d6901a8ab9c9ff43da authored almost 2 years ago by gabl <[email protected]>
Overhaul the documentation to remove all references of the ArchiveTeam instance
cddce9230edc42edfdb844d64333e7328f98110e authored about 2 years ago by JustAnotherArchivist <[email protected]>This makes the docs more relevant to anyone wanting to run their own instance of ArchiveBot.
It...
ca550b992db6e3d8c27a3a3e9fa6b891e6b03987 authored about 2 years ago by JustAnotherArchivist <[email protected]>f2743e2c3d557b95f3346c131e62fc7dd58979a2 authored about 2 years ago by JustAnotherArchivist <[email protected]>
Add additional github ignores
38c67e162439fa9dbb82babc65393a5d77f064fe authored about 2 years ago by JustAnotherArchivist <[email protected]>Globally ignore s7.addthis.com
aeea17002409ed6bf43ea25f90318b0541dec978 authored about 2 years ago by JustAnotherArchivist <[email protected]>This appears on a lot of sites, and almost always times out, slowing down jobs. Other addthis.co...
28d1f40b1d31cb11b147d773f644d61972e2fa8e authored about 2 years ago by Pokechu22 <[email protected]>Both of these are 406.
385bcd795f820400929f7eb22603f9e0f08c30cc authored over 2 years ago by Pokechu22 <[email protected]>Add 2NT's blogging platform to fc2-blog (same backend software as FC2)
ba5a520c21261f19e469a35555b3a2be93ceef09 authored over 2 years ago by JustAnotherArchivist <[email protected]>9be05e1e2f82fb83abe318fd73305cbcddf7dddf authored over 2 years ago by JustAnotherArchivist <[email protected]>
Add Arabic MediaWiki igset
5c8c7ecafa1d4754ca450b4f41a11caccdeec0e4 authored over 2 years ago by JustAnotherArchivist <[email protected]>970732901f7d41d74d1094fcc03322eb99e7667e authored over 2 years ago by JustAnotherArchivist <[email protected]>
Update IGN ignore in badvideos igset
fc108288080954c1d7d70ed67acb15d0b138ad5f authored over 2 years ago by JustAnotherArchivist <[email protected]>8ccaff69f76f709d5c67f3991cfd039a600927d6 authored over 2 years ago by JustAnotherArchivist <[email protected]>
Global ignores: Ignore Discord assets
4a672dbff49597dd8a1f53d95ee60f6ff17a5c87 authored over 2 years ago by Sanqui <[email protected]>
Discord invites show up in a large portion of the Web nowadays and
hog jobs with 100+ partially ...
dateIssued_page seems to be entirely broken on many instances (or in general?), but more than 10...
a84a6e93beb16a69299ef8d384b01b8dc871b3ec authored almost 3 years ago by JustAnotherArchivist <[email protected]>/discover is for the XMLUI, /simple-search for the JSPUI.
a857ebddc4e988047b51df3baf6d78b6f37da9da authored almost 3 years ago by JustAnotherArchivist <[email protected]>9c9c84e2d6dbdc043ce59cb9d32779aa8d4b520d authored almost 3 years ago by JustAnotherArchivist <[email protected]>
ef97da7729ad689dcf625eb1be59c3380927fd4f authored almost 3 years ago by Ivan Kozik <[email protected]>
da1bcdd326b2b3f5957eb282e1cef3070cdd5b6f authored almost 3 years ago by Ivan Kozik <[email protected]>
Fix compatibility with PyYAML 6.0 (mandatory `Loader`)
1cab84619ffd21a9e3df36b3b0bf829777998af4 authored almost 3 years ago by JustAnotherArchivist <[email protected]>e050864d333e4b332a21671cb5f08f2ffd9172fe authored almost 3 years ago by JustAnotherArchivist <[email protected]>
Add notweets igset
22b3b08e18c5f15113a95c94bdf6cd3921b0dbd6 authored almost 3 years ago by JustAnotherArchivist <[email protected]>880bb95ed13fe0685599a317a5c6e4fed9098499 authored almost 3 years ago by JustAnotherArchivist <[email protected]>
ignoresets: add a few forum URL patterns
16be7656148451cd1d22101986e63b2190222ad6 authored over 3 years ago by Falcon Darkstar Momot <[email protected]>Be looser on singletumblr igset
c94b1a26862c55475185f46839ce62de1101a140 authored over 3 years ago by JustAnotherArchivist <[email protected]>This lets jobs grab video iframes and offsite links that aren't Tumblr blogs (except custom doma...
2b2fde86863003e52aae985d5a80d0e7d86a7cd0 authored over 3 years ago by JustAnotherArchivist <[email protected]>Add !reason alias for !explain and --reason for --explain
0522ca3b81ce2f418f593620fb2a58bf74cd88c5 authored over 3 years ago by JustAnotherArchivist <[email protected]>aa4e4807dc1528698870fe58e49d1f725729c140 authored over 3 years ago by JustAnotherArchivist <[email protected]>
Fix dashboard references from EFnet to hackint
e964397b48ee0a8f8df4e4ae38f49d0acea25f8b authored over 3 years ago by JustAnotherArchivist <[email protected]>0da5be4e2e41e1cd1283f3475f263de0c15601bf authored over 3 years ago by JustAnotherArchivist <[email protected]>
3dcd873f2f7d741a979aa110cba75a39e308c0dd authored over 3 years ago by Sanqui <[email protected]>
Add counters for status codes and queued/downloaded to finished page
c1020e56bcd3d37596ff7695f6503e76aa4e0bca authored over 3 years ago by JustAnotherArchivist <[email protected]>48e8b12a923587d4492de46ca43ea1eef46ba5bf authored over 3 years ago by JustAnotherArchivist <[email protected]>
Add Japanese MediaWiki igset
d3d10ceb2a7258acffe9a81d1ae0e0e54911b1d0 authored over 3 years ago by JustAnotherArchivist <[email protected]>57c8b061ed709c6cfe21f1d4bf4dfc08cb7c7414 authored over 3 years ago by JustAnotherArchivist <[email protected]>
Add a hacky script for clearing a job's cookie jar
877335ef4b5ebf2913609ce326873df41abeb392 authored almost 4 years ago by JustAnotherArchivist <[email protected]>Cf. https://github.com/ArchiveTeam/wpull/issues/448
Based on https://gist.github.com/JustAnothe...
39e097c4edbc0c3eb3b8d0269d85666272531091 authored almost 4 years ago by JustAnotherArchivist <[email protected]>Pin SQLAlchemy version to <1.4
f163982afa9a0d305ef016c2680e11d5a4221846 authored almost 4 years ago by JustAnotherArchivist <[email protected]>https://github.com/ArchiveTeam/wpull/issues/463
e9354853ad479519b30e2c42b6cba1c12c7ff1ff authored almost 4 years ago by JustAnotherArchivist <[email protected]>Fix get-pip.py URL for Python 3.5 tests
3bd500fcb34b9847dac266ec71a10dff4e24097d authored almost 4 years ago by JustAnotherArchivist <[email protected]>Cf. https://github.com/pypa/get-pip/issues/61
891ae4f48675b28017c7631a938fb9bbb08aaad1 authored almost 4 years ago by JustAnotherArchivist <[email protected]>Add curl user agent
2d87c6b7534a62f418cca5b4b857ddf9cfe52f8a authored almost 4 years ago by JustAnotherArchivist <[email protected]>515a6aa50601c69ebe2c6869b68db5682f9962e4 authored almost 4 years ago by JustAnotherArchivist <[email protected]>
Load igsets and UAs into CouchDB on cogs start
ed1feffa53a9dec2029ce8a14cd4d20e13673a61 authored almost 4 years ago by JustAnotherArchivist <[email protected]>53872f629472578c1863c761df577772eb03b8f7 authored almost 4 years ago by JustAnotherArchivist <[email protected]>
Switch back to canonical wpull repository
d21680ebbc32d3eebe25117eefef857f0831806d authored almost 4 years ago by JustAnotherArchivist <[email protected]>Bump dnspython to 1.16.0
f1ba36e2903efe9e8bd5804a39d440eb74f5fcb6 authored almost 4 years ago by JustAnotherArchivist <[email protected]>Stop filtering the wpull environment
65aab2462c02f01c2d920848c226057b86e00f2a authored almost 4 years ago by JustAnotherArchivist <[email protected]>All set environment variables will now be passed to wpull, including for example LD_LIBRARY_PATH...
7a68211b334307da13f3a94c0e7baa382d057da8 authored almost 4 years ago by Falcon Darkstar Momot <[email protected]>8057afd0dc20b58c0ac5a7fba3ce77d41bdea300 authored almost 4 years ago by JustAnotherArchivist <[email protected]>
ab15e7a5c2a2fec90abfd446c2281b2bcbbabce8 authored almost 4 years ago by JustAnotherArchivist <[email protected]>
wpull depends on dnspython3, which is only a transitional package nowadays and hasn't been updat...
09c0de722e7f6dd5e6c14f4e4518bde4016d769a authored almost 4 years ago by JustAnotherArchivist <[email protected]>Update openssl-less-secure.cnf
47797be730d1aaf57361497bd516135c164da8f9 authored almost 4 years ago by Falcon Darkstar Momot <[email protected]>Update ciphers from DEFAULT to ALL since we intend to accept weak ciphers
c605b6eab18cdc81804f13bdd1cf507795d9e8e6 authored almost 4 years ago by Falcon Darkstar Momot <[email protected]>Fix ignores sometimes not being applied correctly due to thread-related race conditions
3fbf32bb3d2c9348527c700e8ccfab093c80c4f2 authored almost 4 years ago by JustAnotherArchivist <[email protected]>Fix Python 3.5 tests due to current pip no longer supporting that version
7e57a6107b7634d0c3cc9bedb73d09427ff99dbe authored almost 4 years ago by JustAnotherArchivist <[email protected]>b37eab3c3c46a9b2540d3522169e9f7f5f5a53df authored almost 4 years ago by JustAnotherArchivist <[email protected]>
Add IGN and MLB to badvideos
f32be97154451d5de9f42f113708059832fb5969 authored almost 4 years ago by JustAnotherArchivist <[email protected]>6344f443cf27a2a57d7d68aec01d70a4fbd5d9aa authored almost 4 years ago by JustAnotherArchivist <[email protected]>
`set_patterns` is called by `ListenerWorkerThread` and modifies variables used by the main wpull...
4ce74e6f8894ab271fe91e81e7df2e40f0af666a authored almost 4 years ago by JustAnotherArchivist <[email protected]>Pass TMPDIR to wpull if present to avoid jobs stalling on systems with small /tmp partition
3af42e74b7eeb9aced663cdba0eb700a068ec497 authored almost 4 years ago by JustAnotherArchivist <[email protected]>e303082ae7d2b6c3ac845698b80e637ccb46e825 authored almost 4 years ago by JustAnotherArchivist <[email protected]>
Pipeline preflight test improvements
0353b943cd734a6282c677997c6494c25d31c6f5 authored almost 4 years ago by JustAnotherArchivist <[email protected]>cd1d4138d8fcd3460700f1ff1844e09d9c7b4f83 authored almost 4 years ago by JustAnotherArchivist <[email protected]>
3ce5fc0691d33fc87400b22b51cd75900087b894 authored almost 4 years ago by JustAnotherArchivist <[email protected]>
Previously, the item directory would be created in TMPDIR (usually /tmp), which may be on a diff...
b0867451f6c6f41089e6e939fb9672d480f5f187 authored almost 4 years ago by JustAnotherArchivist <[email protected]>Typically movie trailers, e.g.
https://video.csfd.cz/files/videos/157/765/157765147/164315717...
80af9d17bc899969d3f23de5c7269b9576c43673 authored about 4 years ago by Sanqui <[email protected]>dfdba96f59747fce86ce50377c3ad108971f5afc authored about 4 years ago by Falcon Momot <[email protected]>
pipeline crashes
71a57558749df02b4c6d9a32f372c259465c9040 authored about 4 years ago by Falcon Momot <[email protected]>Add Korean MediaWiki igset
72d2d09cc447fac0a8acc8feff714b84f16f6e56 authored about 4 years ago by JustAnotherArchivist <[email protected]>Closes #488
c818e3a4b983775a64bb6a9e34a6fffac7a056aa authored about 4 years ago by JustAnotherArchivist <[email protected]>Order dashboard finished page by completion date by default
a3fa117a01cc2c4cb24948b2574b829a2ffb990a authored about 4 years ago by JustAnotherArchivist <[email protected]>31c13ea408b225b0dd9e8772c062a41c7b5a2514 authored about 4 years ago by JustAnotherArchivist <[email protected]>
Improve performance of dashboard finished endpoint
d06fa2f4f20c4cca927247c1919ad8361537e6f5 authored about 4 years ago by JustAnotherArchivist <[email protected]>a88f27d342d5516301f99172530c9ec9c8282e8b authored about 4 years ago by JustAnotherArchivist <[email protected]>
Job.from_ident first fetches the URL and then retrieves the rest of the data (in amplify). That ...
420891bdaf0ef81b3dcbedd8164adfc42f9fbeda authored about 4 years ago by JustAnotherArchivist <[email protected]>Improvements to the list of recently finished jobs
a889ea682449dd3e2f21eabab9a66b8841c03de1 authored about 4 years ago by JustAnotherArchivist <[email protected]>0828ba6a667d02485bcdb16b6f662d53b64fac4a authored about 4 years ago by JustAnotherArchivist <[email protected]>
8321b96cdcb44f2efaeec8ce70b007afa6f6f6b1 authored about 4 years ago by JustAnotherArchivist <[email protected]>
Add list of recently finished jobs to dashboard
52632ec8b47139cefe2453921b96eded3069f01e authored about 4 years ago by JustAnotherArchivist <[email protected]>