github.com/ArchiveTeam/twitpic-grab commits | Ecosyste.ms: OpenCollective

twitpic.lua: match > gmatch

9303ec6bd0d4848c8ec366e452a2e7f0b052a021 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: skip comments pages for now

ad11fde30110ba82ac5cd7038d562bd42929249c authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: fix

c407ce74e0f75cbe363550608d1b95152b8e3109 authored over 10 years ago by Arkiver2 <[email protected]>

Revert "twitpic.lua: fix for comments pages download"

This reverts commit 2fe0d198d8772a7ae3521aaaa4967bae2c3cd20f.

6ee62c648944cf7a5de2818a8060370746352863 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: fix for comments pages download

2fe0d198d8772a7ae3521aaaa4967bae2c3cd20f authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: do not generate page on 404

b0e6242439a00b4c19f4143c17cf308fa9b6fcf1 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: fix for infinite page downloading

5d06140a841c73b26c1b5e2fc7c382218db3a23d authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: add support for the JSON comments

and check is the comment JSON is already downloaded.

f0b5c0d0606d61ed5430413b569e06f408086944 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: update to downloading comment pages

a38d06bdda4fca64921767731c2dc8aed3a01840 authored over 10 years ago by Arkiver2 <[email protected]>

pipeline.py: support 5 urls in 'image'

01fc18d4437f000e5d882de4e596831f302c20ca authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: add local media_id

2990021fcc2e589608f5a142cd058e7c56b3ae80 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: add comments page url base check

500c6398e47d329975a451ae7b3301c3d41877c8 authored over 10 years ago by Arkiver2 <[email protected]>

pipeline.py: add comment page url

f8edef4db94f1d4d724df711fdece5d95999b244 authored over 10 years ago by Arkiver2 <[email protected]>

pipeline.py: bump version

ac50a030c2e145e0306e40fd1313e6c458ff9d1b authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: fix

96df33297f084426019ad7ab1a5ee3020472a064 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: fix

5ce33fd9e373962fa365e8c3e2b60d8a49d8a395 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: fix itemid not defined

959e196bfcf5b64a436902740bd9d7b35d8859f1 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lia: fix item not defined

0a658549956dd14d4bc6013482187707d965d030 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: update

3082a54f3b26f8fbdfd2c7ce4788f95d6a0e24fa authored over 10 years ago by Arkiver2 <[email protected]>

pipeline.py: add all urls from API. add support

for 'event' (will maybe be used in the future). bump version.

c5ed844c06cc718761b94f74e94fea9a7fe0b91a authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: another try for the comments

901752a155363e27c8d661306301a85e4afde32c authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: comments try

0c2879b267e790bbf71363ba5158549d6880bbc9 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: comments fix

dfee59342965b882b74eb234d1ec36b5323cab2b authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: comments

fb56af09a36d1b86494e67de99ded50f215a5179 authored over 10 years ago by Arkiver2 <[email protected]>

Revert "twitpic.lua: comments"

This reverts commit fa873a9ea2af390418e3a47f58ec90fad3b06ec2.

1236cef07bb46babf39c744853a55e9f8c62619b authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: comments

fa873a9ea2af390418e3a47f58ec90fad3b06ec2 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua

b69b2b258cd20206cc497ea1e19980be3007ea9e authored over 10 years ago by Arkiver2 <[email protected]>

pipeline.py: bump version

930a44fcba2bd9a0c3358eb834df2495cefb64fc authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: do not check for previous pages in

tags

ed0d3a95725d21bc88d5a2f7616d3900c03634bc authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: fix typo

d49e0b638cd87c90a87b60107a07d28d73ee69ca authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: trash piece of script

ac66cb41cc61590016f7dc87e74afc8eea5ed831 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: more locals

76c590d00011af7895a503b4762046c3242c03c4 authored over 10 years ago by Arkiver2 <[email protected]>

pipeline.py: bump version

8607af553b3b2de6521a1aaadf4265be3ca3c65e authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua

e6ed98fc0c540acf5093bd0b0095b10cc3d562ae authored over 10 years ago by Arkiver2 <[email protected]>

pipeline.py: bump version

9ead1c25c26316eac4f4ad48c30f24dfdfd60f1e authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua always load as html

8ccf3e95b290ecc77fe0faa25f9d37b6bb27f0bc authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: better support for comments scraping

4a377b4a442cdede3f5c79ec993e76c15a02c512 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: fix error

d5c00bf22cb8468ea244121fde1a347ca1989923 authored over 10 years ago by Arkiver2 <[email protected]>

Revert "twitpic.lua: better url scraping"

This reverts commit 1c3f802ac9b5bcd6b6e25c2d5f5d11fe2a64793a.

fae9677eadd40f4e76e65c5318f9d10f98b0195e authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: better url scraping

1c3f802ac9b5bcd6b6e25c2d5f5d11fe2a64793a authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: url scraping for comments and

and more imageurls

8058abf098fafdcbe8ad1a7ee7489aa549afb06c authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: fix url scraping for pages in tags

30a2beb44c9fea7661c4efa5470f7e2ffa0152dd authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: add support for pages in tags

through custom url scraping

6e2fa966639d4673698deb24609a25ef477ee715 authored over 10 years ago by Arkiver2 <[email protected]>

Merge pull request #3 from ArchiveTeam/report-failure

Report failures to remote collector

bcda09818e94f603de1adc77a21de414521ae298 authored over 10 years ago by Arkiver2 <[email protected]>

Merge pull request #2 from anthonyeden/master

Create README.md

6fdd28dbca88908d1cdc16f2f871da775993a89d authored over 10 years ago by Arkiver2 <[email protected]>

Merge pull request #4 from ArchiveTeam/manual-tracking

twitpic.lua: track downloaded URLs in the Lua script

16c06be0fe5236d66222d9a3b7a7a9bcdfa3b2be authored over 10 years ago by Arkiver2 <[email protected]>

Track downloaded URLs in the Lua script.

Wget seems to track downloaded URLs on a per-source URL basis, which
means that we end up downlo...

5ae672a44fb3206b4aa8b1c75889271efb1285dc authored over 10 years ago by David Yip <[email protected]>

Accept wget invocations that had network and protocol errors.

We're recording these via the Lua script; therefore, we'll know which
URLs issued network/protoc...

a6135a953fac5b08e1666e080ab202deb0700a8e authored over 10 years ago by David Yip <[email protected]>

Report failures to a remote server.

A "failure" is defined using response codes. The following codes are
treated as failure codes:
...

b303d6eabdc5fcb5a9b4363aff1a6099636754ed authored over 10 years ago by David Yip <[email protected]>

twitpic.lua: exclude all links with '%22' and '"'

a39c839500eca59fe844c4747ac41e911074179c authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: add support for pages in tags

d957cdc97f528545d10402ccfef6341d1627b143 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: fix to download videos

efd20db318cbc034e58eb55c0dfef4538f6be5ef authored over 10 years ago by Arkiver2 <[email protected]>

pipeline.py: bump version

0f8cebeb42e073edc47e77c2b462bffe79c520c3 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: check if link are already downloaded

ab732af1496eeced32c71427456dd5413ef06d31 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: exclude all links with '%22' and '"'

70937fb67b57423265be5870a910694c776338b1 authored over 10 years ago by Arkiver2 <[email protected]>

pipeline.py: bump version

cde2168624abcaa7b5961017c167d3d2f5a70f7e authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: exclude all links with '%22' and '"'

6d91aa61ba40558d4aa7c0ab47f79104ef0dd7d8 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: exclude all links with '%22' and '"'

for all image: links

e175e7ee958b5a50b576ae0f91b9b7d4a7235473 authored over 10 years ago by Arkiver2 <[email protected]>

pipeline.py: bump version

142f0e0d1e4bf3c50555540d33514182281e7f0a authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: exclude all links with '%22' and '"'

967fba07c88b497a2b2e9877d06b8c6631260bd7 authored over 10 years ago by Arkiver2 <[email protected]>

Remove unused yajl-ruby dependency.

2147b4e577fcca7998774ad419be8dfcf3dba46d authored over 10 years ago by David Yip <[email protected]>

Failure server: use tai64n timestamps in document IDs.

Why? I have an irrational dislike of periods in my document IDs.

0d3da31bcaf16df2c79a7fd2f2123f874b6a0b39 authored over 10 years ago by David Yip <[email protected]>

twitpic.lua: match > gmatch

9dcfa8cfca640a1ec4dfe84b2dc8540fa813fdd1 authored over 10 years ago by Arkiver2 <[email protected]>

Spike out a server to report and store failures.

e80368d76e27685fe6020156cadb0d5ecd2efe0c authored over 10 years ago by David Yip <[email protected]>

twitpic.lua: fix again for url scraping

be9323f0d6f47e4ce48a2d8f30d6b5ab1ad4424d authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: small fix in custom url scraping

7633c72b935eba10863a85255bcc691da0efd7f1 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: small fix

0384f046d6b6530906c06e27d7a8801e8cd02ee3 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: forgot one and

9f7a438e82d73b1f5b3d97b30776bb39ce09ced3 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: make sure the extracted url

start with html

f95026995b49f32d7c40986de9af3d84bc49c900 authored over 10 years ago by Arkiver2 <[email protected]>

pipeline.py: bump version

e4539ebfa929e9adc64ce1cdb7e2fc1d6aeb558a authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: show all lines

b87568993b4c2c98255902376716f42966a14a97 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: fix for pattern matching

696a57f38ecab3b69b11f077e75b3289609ae768 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: add scraping for video urls

125b48075e696d78e7b500fa2646e29884c48879 authored over 10 years ago by Arkiver2 <[email protected]>

add files for custom url scraping

acc92de6535e4853a4caa5fce984518e1c438a9d authored over 10 years ago by Arkiver2 <[email protected]>

pipeline.py: bump version

0e06b701f7441313b6939d675490b5285852b416 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: prepare for tag and user test

055b6de321ef994a26df35e58dc1db14fc42aa88 authored over 10 years ago by Arkiver2 <[email protected]>

Create README.md

7f431748ba9bd62a6d413a77518673c47ee68209 authored over 10 years ago by Anthony Eden <[email protected]>

pipeline.py: bump version

ed8c7be12646235758d1e94e5dc8d119e9ba5e30 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: return true on all with item value

ed3b505b75a166814a783242d2d7269cd1762c37 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: remove not needed excludes

734e5c10d055dce16b667f067d7b4fbf36c85e07 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua return true > verdict

c3adc5a97ab77a0480e8c7c11626b3cf2f66636c authored over 10 years ago by Arkiver2 <[email protected]>

pipeline.py: bump version

46fcef1b9519cb93100878514075336d7e1520a6 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: fix to download videos

addcea8e1d12bcec4188c8b6533180d23275e0f3 authored over 10 years ago by Arkiver2 <[email protected]>

pipeline.py: bump version

1ded435c70efd7d56799d861d7fee730f4b9949a authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: undo a previous edit

82b36424966ed74aa36f95eaa43b48f54ce43776 authored over 10 years ago by Arkiver2 <[email protected]>

pipeline.py: bump version

319023f9a4fc4cac57c191cd4c792dba089cb828 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: oops, added then instead of and

4b559ea862af7df21bbab11e82219a8d5a39fcb6 authored over 10 years ago by Arkiver2 <[email protected]>

pipeline.py: bump version

a55daed5e74786ac88089a175ed4415ad3490f24 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: rules for status code 0

05a1b10fdedbe2237d3ce7273b62a9a56686bb14 authored over 10 years ago by Arkiver2 <[email protected]>

pipeline.py: bump version

0e7d232c02446a70e591cb305b3ebed0b67b01a8 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: download if not downloaded yet

9b00bafe9515748bc24ac763454a80b37c39a03f authored over 10 years ago by Arkiver2 <[email protected]>

pipeline.py: bump version

06b158dba73bd3f64699cc95158a058d6dea7a9a authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: fixes for what needs to download

2f5c5e1eca480f4382c2b951f6ce58ca8dafbd79 authored over 10 years ago by Arkiver2 <[email protected]>

pipeline.py: bump version

00dfb4da8ffdb32f67375950a875fe9853f4a9be authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: do not download external urls

f282803e3c287e4a2d329ef0c0d14fca35463271 authored over 10 years ago by Arkiver2 <[email protected]>

pipeline.py: bump version

419fc8cc57292239055f7f54d5001892c3e7f156 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: do not retry 404 and 403

db7917f0c8aecb9457a4d623e43ddab87f5a19e1 authored over 10 years ago by Arkiver2 <[email protected]>

pipeline.py: bump version

87c0e0b804e8f51dcd70a1968cc5e8a1cd5f2921 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: exclude links like

twitpic.com/eaag80/%5C%22%5C/images%5C/hud-img-arrow-cc.png%5C%22

7c9110889e972c6123056382bb1f1935e3cb3985 authored over 10 years ago by Arkiver2 <[email protected]>

twitpic.lua: sleep time to 0. go fast

1cac34993aa3851f43661c74a08262d1f3eedaa8 authored over 10 years ago by Arkiver2 <[email protected]>