github.com/ArchiveTeam/ArchiveBot commits | Ecosyste.ms: OpenCollective

pipeline: Make youtube-dl work even when ~/.local/bin is not in PATH

68474dcb4ff18656da260e15f73e8e28d3e1eb99 authored over 9 years ago by Ivan Kozik <[email protected]>

dashboard: Fix comment

0c752ea783ba2756e92a5f36bc8ee9ac825c2b44 authored over 9 years ago by Ivan Kozik <[email protected]>

doc: Note that URLs are case-sensitive

[ci skip]

f23f9212c3a574034e04495103e349c09b5ae2ae authored over 9 years ago by Christopher Foo <[email protected]>

Don't tweet aborts because they don't mean much

22d0bb7325b3adec7e0f5970f85fef08842d9da9 authored over 9 years ago by Ivan Kozik <[email protected]>

Go versionless on the dashboard

fc0a2d0aed9c63c7d8c43b829c19e50f2b1e2af3 authored over 9 years ago by Ivan Kozik <[email protected]>

Move 3.0 dashboard link to help text, to make header fit in FF

800749ecaebe74c10de347a84aa493ded8b28913 authored over 9 years ago by Ivan Kozik <[email protected]>

Enable the custom context menu in Firefox 41+

18c55f5ab83216d3c5617091ede7b230ef3ee09f authored over 9 years ago by Ivan Kozik <[email protected]>

Bump pipeline version

0ee7f4999a305bb0cd614f5e454786bce5c80923 authored over 9 years ago by Ivan Kozik <[email protected]>

Merge pull request #170 from 12As/report-pipeline-status

Report pipeline status (running or stopping)

e7a520405796df2d9d81bab89c58591887f6042c authored over 9 years ago by Ivan Kozik <[email protected]>

Fix event handler syntax for pipeline status

cf31b29a17566933099658435d71a648694e06f3 authored over 9 years ago by 12As <[email protected]>

Add pipeline status to pipeline dashboard

b28e5ccfa709151219adece7d802238cd1e5ed60 authored over 9 years ago by 12As <[email protected]>

Add status to pipeline process reports

f51444092949d5b45f1fc5f935667b9d9f086c67 authored over 9 years ago by 12As <[email protected]>

Running/Stopping status events added

83ebb363d8bc5ef87bf2d418cb1317b398b636c5 authored over 9 years ago by 12As <[email protected]>

Revert "Temporarily ignore voat.co, as it is not responding"

This reverts commit f6fb34ad5b46cf730d5e07475b1c1fc73b3570a8.

voat.co is back up.

091598b1d2eda4b78a829bf1f2822816d44282fc authored over 9 years ago by David Yip <[email protected]>

Add .kr TLD for blogspot

8cebb0f34d5472ac9fd762a86f67a25326708293 authored over 9 years ago by Ivan Kozik <[email protected]>

Copy youtube-dl bin so that it doesn't need to be in PATH

85bf3abe929e6b0e80c51d9a586d1f7169d31ec3 authored over 9 years ago by Ivan Kozik <[email protected]>

viewer: Add data cached disclaimer

[ci skip]

b1ba5c2a4b5c1f5db3e4c7d10936be95d5a2fe5b authored over 9 years ago by Christopher Foo <[email protected]>

viewer: Fix no search results text output check

[ci skip]

333cee71b6cc6c8ebaf1591579fdbf1158824be3 authored over 9 years ago by Christopher Foo <[email protected]>

Merge branch 'viewer-download-links'

b45b58650f0c447cf7ce3f062e0bcc0711cea776 authored over 9 years ago by Christopher Foo <[email protected]>

Fix use of ARGV - ARGV[0] is first arg, not program name

76ce6565cd9f9bc4b239b5b43d8fc6bac2e19375 authored over 9 years ago by Ivan Kozik <[email protected]>

Ignore simple.reddit.com

a6ce31540b944bf9ea38d3e365b4669c04ab5b52 authored over 9 years ago by Ivan Kozik <[email protected]>

Ignore /.mobile on reddit

c70d9724fa228ff19d1f45782ca1d45966af747f authored over 9 years ago by Ivan Kozik <[email protected]>

Temporarily ignore voat.co, as it is not responding

Please revert this when it comes back up

f6fb34ad5b46cf730d5e07475b1c1fc73b3570a8 authored over 9 years ago by Ivan Kozik <[email protected]>

Fix styling when nicks are not shown

11a25a43236039481353282bfb4f46fbf3b2b82d authored over 9 years ago by Ivan Kozik <[email protected]>

viewer: Add download-from-IA links.

39ae8260bd358c1ce2f29de103a2dad5fbc79b7c authored over 9 years ago by David Yip <[email protected]>

viewer: Add requirements.txt for pip -r convenience.

7037bd92138b2ae9a44e1b8f1651947517eb708d authored over 9 years ago by David Yip <[email protected]>

Ignore another streaming site

f07f9c93e3452f434ffbd91c5ba0bf1f336e4432 authored over 9 years ago by Ivan Kozik <[email protected]>

Don't treat Logging error as fatal error

Maybe revert this when https://github.com/chfoo/wpull/issues/268
is fixed

ac2b8211fefad908e12af4f9d2dd723b42dd9a05 authored over 9 years ago by Ivan Kozik <[email protected]>

Ignore another share link

e026f2293577a715664ffae05a9d93f74a8747e8 authored over 9 years ago by Ivan Kozik <[email protected]>

Remove some unnecessary authorized? checks

f4c431a56be88e3fe3e67a99de4072ab5a2c37b2 authored over 9 years ago by Ivan Kozik <[email protected]>

Allow unvoiced users to do !ao

ca39708588c7262eec2c3100ff01b65aa580a2d6 authored over 9 years ago by Ivan Kozik <[email protected]>

Ignore another streaming site

833ba2f0f9e0738aa4d6e6f74d364c349cf17bfb authored over 9 years ago by Ivan Kozik <[email protected]>

Fix filename

4351902a6af62e903570f53ed669e8205e1f43f3 authored over 9 years ago by Ivan Kozik <[email protected]>

Add noonion ignore set to ignore .onion sites

c177cd6144ee9cba31d0511e26a05491e166532a authored over 9 years ago by Ivan Kozik <[email protected]>

Revert "Use better font-size in Firefox"

This reverts commit 66533bf58984e65e215d0fab2a9d9b5c1627b32c.

Well, that was embarassing. My p...

043f0848d04830c99d9d37f0f7cd51150a26d719 authored over 9 years ago by Ivan Kozik <[email protected]>

Use better font-size in Firefox

66533bf58984e65e215d0fab2a9d9b5c1627b32c authored over 9 years ago by Ivan Kozik <[email protected]>

Enable custom context menu in IE10 and IE11

7a655a1af870664db1a1f4c1eb4dcfc569a7aef0 authored over 9 years ago by Ivan Kozik <[email protected]>

Use Addressable::URI to handle IDNs. Closes #165.

41d6f3207189b760eb859a5528b3ba2009760eb1 authored over 9 years ago by David Yip <[email protected]>

dashboard: Stable sort for /ignores/IDENT,...

ff030ada38e1b3c1fa745579a31ba95103c7d45e authored over 9 years ago by David Yip <[email protected]>

dashboard: Add /ignores/IDENT1,IDENT2,...,IDENTN.

This resource returns the ignore patterns for each job. The response is
of type text/plain. Ea...

e7859f2732e2cc3d95ed2ac1a535eafb5376af1b authored over 9 years ago by David Yip <[email protected]>

Ignore another share link

61330fb7cd7c789c1a8fa2d099ea0e6358cda09a authored over 9 years ago by Ivan Kozik <[email protected]>

Ignore Yahoo beacon

4d5a160ac267bebb37c6cee0387210028edf8acf authored over 9 years ago by Ivan Kozik <[email protected]>

bot: Don't announce global ignore sets.

Everyone knows this set is added, so we can save some lines (and cut
down on noise) by not broad...

62fe4afbd2bc99de3a93bdc45c33ba48fbd5008c authored over 9 years ago by David Yip <[email protected]>

bot: Link to "youtube-dl is totes experimental" blurb.

634ab3c4f4a640274c4fe7ba55dd8c84c47dd921 authored over 9 years ago by David Yip <[email protected]>

Bump pipeline version. Closes #82.

Historical (irrelevant?) note: I'm doing this pipeline release while
waiting for an anime conven...

92ad6f3d084e9e571c9eaf4dc1304e26d27135b0 authored over 9 years ago by David Yip <[email protected]>

pipeline: Use wpull 1.2.1. #82.

ee73277aafe6994b44d5aa41fca1e291fcee177c authored over 9 years ago by David Yip <[email protected]>

pipeline: Switch to chfoo/wpull@develop for youtube-dl tweaks.

Specifically, chfoo/wpull@develop passes IPv4-only and TLS cert check
skipping to youtube-dl.

c46fca27d75817ffd41778ad30e5e1b14394be6b authored over 9 years ago by David Yip <[email protected]>

pipeline: Add youtube-dl to egg list. #7.

bee7f6c403d9c817d000647322a076ee6a58d684 authored over 9 years ago by David Yip <[email protected]>

pipeline: Tell wpull to use --youtube-dl when the job wants it. #82.

a075f8c100d71d7122450428e3481d9a5a8df6d4 authored over 9 years ago by David Yip <[email protected]>

pipeline: Add NCR=1 for job URL with blogspot

re #80

abfc08f9160c3de1dcaece542ab3011b17cd59c2 authored over 9 years ago by Christopher Foo <[email protected]>

pipeline: Raise disk/mem monitor to 500m/50m. Set max redirect to 8.

Raise the monitor disk and memory args to 500 MB and 50 MB respectively.

Set the max redirect t...

1838f2f1709ba74a4777f25e3772635a7de6170c authored over 9 years ago by Christopher Foo <[email protected]>

doc: Reposition youtube-dl version notice, take 2

bf5d5754eccb4857daf57df04a40aaa2542d06de authored over 9 years ago by David Yip <[email protected]>

doc: Move youtube-dl version notice closer to option summary.

[ci skip]

00ac07033960563e442b4495f748558c4c864ebd authored over 9 years ago by David Yip <[email protected]>

doc: Document version restriction for youtube-dl. #82.

c3d91819600226f66474b14e7be37bd5cc8dc74e authored over 9 years ago by David Yip <[email protected]>

doc: The 32-character note limit no longer exists.

996698e8bd70d81dd6b5296a439d8bf43a530001 authored over 9 years ago by David Yip <[email protected]>

doc: Strip trailing whitespaces in commands.rst.

0840d082369e51137f76cd8e235c90214e7ff9db authored over 9 years ago by David Yip <[email protected]>

Document --youtube-dl. #82.

31c373d7311753422c6139ca7700f026366bec58 authored over 9 years ago by David Yip <[email protected]>

bot: Disallow recursive youtube-dl. #82.

A test job on Youtube[1] had youtube-dl crawling channel pages and video
pages; it ended up down...

48d9fe585c8530dcc2d0a9bde8c5e81c763ad2be authored over 9 years ago by David Yip <[email protected]>

bot: Don't show job options if there weren't any.

697ad73496f26c13c7aa7afe0ecc02ddee7db2ae authored over 9 years ago by David Yip <[email protected]>

bot: Add youtube-dl message; shorten up status messages. #82.

4a8a22aef56e0e531454e21fbe6031382bf56d1a authored over 9 years ago by David Yip <[email protected]>

bot: Recognize --youtube-dl, pass it to the job. #82.

a1534f81e804c8c6feffdcdb07995a61f33705e4 authored over 9 years ago by David Yip <[email protected]>

lib: Remove dead ArchiveUrl* code.

This was used as part of a process that looked for WARCs in ArchiveBot
GO Packs and enhanced Cou...

7d732a7f425950f60e1949e356d9895dd638a867 authored over 9 years ago by David Yip <[email protected]>

Reorder and reword some help text

7787cd03c5f146cd0d69bfe8516aed9eedfe09fa authored over 9 years ago by Ivan Kozik <[email protected]>

Document &moreDom=1

7a665d5533614dafa34284ac13bda95292e505cb authored over 9 years ago by Ivan Kozik <[email protected]>

Ignore more ?sort= pages on reddit

9733c2bf4796737418eac525f96810cef9ffe6f9 authored over 9 years ago by Ivan Kozik <[email protected]>

Ignore another share link

ec09d46718371bff939d031e6e580e0fe97c6ee3 authored over 9 years ago by Ivan Kozik <[email protected]>

Ignore Special:Diff and Special:MobileDiff

a8a7c8958c592a4efb38864ab0385f2455f5a0a4 authored over 9 years ago by Ivan Kozik <[email protected]>

Move file out of _uploading directory if upload fails

259107e10cea2ca1ae85c46f587cc3df1dceda9a authored over 9 years ago by Ivan Kozik <[email protected]>

Ignore another Google Analytics endpoint

e2cb5afcd0d065434b8ca4b013a93472585fb3bc authored over 9 years ago by Ivan Kozik <[email protected]>

minor improvements

9515eff20029eae6d5da059b826d8f6c3646daf8 authored over 9 years ago by Start <[email protected]>

ignore ?sort= for users

5ceb115114384596b66fbcb901182a4a890c42b6 authored over 9 years ago by Start <[email protected]>

Use wpull 1.2

bf020d580b76a45e35615bc5a038d09f3fbd1cab authored over 9 years ago by Ivan Kozik <[email protected]>

Make context menu work with moreDom=1

21772c7930b643b34a82996fad930886b643f785 authored over 9 years ago by Ivan Kozik <[email protected]>

Add a &moreDom=1 option that puts <a> in <div> like before

64579d18472944cba2a46118cb6e1266c3da955e authored over 9 years ago by Ivan Kozik <[email protected]>

Add option for adding text to document.title

42815468ecdda655f18d3e7983c6b5a2939f6249 authored over 9 years ago by Ivan Kozik <[email protected]>

Prevent text selection on context menu items

1bfb6c094bba07242bafa935c37feebed3b47574 authored over 9 years ago by Ivan Kozik <[email protected]>

Remove dubious text selection behavior in stats line

2a6ffdb246808fb6497aa9451b860f7b1278124b authored over 9 years ago by Ivan Kozik <[email protected]>

Close the context menu on mousewheel movement

7f3aafd15e70fa2938c2145b6103c386dc931968 authored over 9 years ago by Ivan Kozik <[email protected]>

Don't word-wrap context menu entries

d23595f19fec40e7f4e27b6879cf50eb4916387d authored over 9 years ago by Ivan Kozik <[email protected]>

Don't generate duplicate !ig suggestion for http://domain.com/

b4b8e93ed55ef3a10df829478a82677a3bc4c0f1 authored over 9 years ago by Ivan Kozik <[email protected]>

In !ig suggestions, show unescaped '-' characters

8be20ea691749bd27e7099c133fc70828ea40a92 authored over 9 years ago by Ivan Kozik <[email protected]>

Escape the path component in the !ig suggestion

9ed351ee932a0f337d904de60888dd27366131a8 authored over 9 years ago by Ivan Kozik <[email protected]>

Open context menu from bottom-left corner when needed

3d418a7cc266d8541bc33c11af0427305ffaf2e9 authored over 9 years ago by Ivan Kozik <[email protected]>

Avoid generating a duplicate !ig suggestion

2547c3020cbc25a9baa05235f420a4869452e4f9 authored over 9 years ago by Ivan Kozik <[email protected]>

Generate more !ig suggestions

2264cf31dcb0b4adbe863a0fb5aaec45eded9380 authored over 9 years ago by Ivan Kozik <[email protected]>

Show ftp:// in !ig suggestion for FTP URLs

bde1f60599a934b5dad7cbd9208d52c159634032 authored over 9 years ago by Ivan Kozik <[email protected]>

doc: Correct table syntax in redis.rst

12aa521719402d5928f2e82110c97f36d952cd52 authored over 9 years ago by yipdw <[email protected]>

doc: First cut at documenting Redis keys and pubsub channels.

6e2d6501d21165e234cf7153f1b5a4f439239e0b authored over 9 years ago by David Yip <[email protected]>

Update user agent strings

bfb0d0081b848decfef4d26e8db90c9850f2275d authored over 9 years ago by Ivan Kozik <[email protected]>

Ignore an Icecast server that doesn't send Icecast headers

f6aa94641f9b1f0d96704110d3401794cf169494 authored over 9 years ago by Ivan Kozik <[email protected]>

Ignore another share link

db4760a059345aba64b104e7c71e07bf1f37b927 authored over 9 years ago by Ivan Kozik <[email protected]>

Ignore m.reddit.com

1432d76151fd1f0bf38877fee368c5763e8b30de authored over 9 years ago by Ivan Kozik <[email protected]>

pipeline: Disallow Python 3.4.0 which may segfault. Bump version.

91e8e311aa7a0ed26c5225931061cc80abaf8c76 authored over 9 years ago by Christopher Foo <[email protected]>

Animate only the height of the log window

8e64f97fb72f0e2a6234ea9ed34eb024f6ac4155 authored over 9 years ago by Ivan Kozik <[email protected]>

Enable custom context menu for Chrome 42+

3490f4a049eadc62316f8ce3619ae4d99f4dd608 authored over 9 years ago by Ivan Kozik <[email protected]>

Document the custom context menu

e6a4d151e2975afda5fbe9ddffe38c3926ee370c authored over 9 years ago by Ivan Kozik <[email protected]>

Merge branch 'context-menu'

0e3a68624533a36bed9c1a74337175957fc912ea authored over 9 years ago by Ivan Kozik <[email protected]>

Adjust context menu item text

656401814426e6013c22d2e83357cf0cfb40bfc7 authored over 9 years ago by Ivan Kozik <[email protected]>

Don't let the log window scroll until the context menu is closed

8a51a24948c0f850460acce29b25d0e357906346 authored over 9 years ago by Ivan Kozik <[email protected]>

Abbreviate the display of suggested commands

b70797678f7b8c5df2692c2f3045d6270a5b8a47 authored over 9 years ago by Ivan Kozik <[email protected]>