Ecosyste.ms: OpenCollective

An open API service for software projects hosted on Open Collective.

github.com/ArchiveTeam/tinyback

A tiny web scraper
https://github.com/ArchiveTeam/tinyback

Services::Isgd: Fix missing character in charset

785482f708d932a497abeb6dd5ed308beb527bc2 authored about 14 years ago by David Triendl <[email protected]>
Reaper: Various speed improvements, write only plaintext

c15a7e5c546895d4c5ac3a61f0a3078c1ade36c8 authored about 14 years ago by David Triendl <[email protected]>
Services: Update advance mechanism to match up with chronomex' scraper

2931748812a842a92eda8699e20fedb4bf1b623b authored about 14 years ago by David Triendl <[email protected]>
Services::Isgd: Gracefully handle service error when no html payload is returned

73285864177881c072a4622b6a9177afbc3451cf authored about 14 years ago by David Triendl <[email protected]>
Services::Isgd: Raise FetchError on unexpected connection close

a45945179b8715ba6b70ad6039172d153035f66a authored about 14 years ago by David Triendl <[email protected]>
Reaper: More aggressive back-off on fetch stall

9e109e76ed217e69a2a9e5d99be45e3442a16cbd authored about 14 years ago by David Triendl <[email protected]>
Test connection keep-alive for Services::Isgd

6f7b47764d20806fc270ea495e146ffbce5c0b24 authored about 14 years ago by David Triendl <[email protected]>
Services::Isgd: Raise BlockedError on 502

96f352346176cbb148aaf571d55e997e3eb38135 authored about 14 years ago by David Triendl <[email protected]>
Services::Isgd: Fix bug in error message

b554e56a0de5330c092ec8bd72ae542f6fae9ccb authored about 14 years ago by David Triendl <[email protected]>
Merge branch 'isgd-blocked'

8ef537de3a414f8109e3cc50358a5b0d18a7eaa8 authored about 14 years ago by David Triendl <[email protected]>
Services::Isgd: Parse URL from "URL blocked" page

7747422aa1c0c782e15965d443a2e5efebf445d5 authored about 14 years ago by David Triendl <[email protected]>
Services::TinyURL: Only do a full GET request if absolutely required

6e742047da8e6eb5278c693bd34e1ce840d18f40 authored about 14 years ago by David Triendl <[email protected]>
Isgd: Add testcase for spam-blocked site

4f866186c6db3cb351bafd6dbf769a2ab84762b5 authored about 14 years ago by David Triendl <[email protected]>
Reaper: Ignore Errno::ETIMEDOUT

2fc9668ab6da34c8b54fc5f4406bbbff904db51e authored about 14 years ago by David Triendl <[email protected]>
Services::TinyURL: Raise BlockedError when no Location is given on 301

395d39d39864aa6b79f53f914efb3c54e8794dee authored about 14 years ago by David Triendl <[email protected]>
Reaper: Reduce logging output

f7fd593d4a2e911c091d466941b77d310b00d4c7 authored about 14 years ago by David Triendl <[email protected]>
Reaper: Fix retry logic, set max number of tries to 3

4d3224632145f38bc3c8a8e7fda5ee53cf11bfc8 authored about 14 years ago by David Triendl <[email protected]>
Reaper: Increase timeout to 30 seconds

bd1e76f6b8dc7d57cc75d2fe986aab26a804cd76 authored about 14 years ago by David Triendl <[email protected]>
Reaper: Recycle service and retry on FetchError

5d95822c91106e741929972de668557a886e79d1 authored about 14 years ago by David Triendl <[email protected]>
Servies::TinyURL: Spread load evenly across all servers

b16800c4d4fc03f137c795ba08d430bfa8d6d33d authored about 14 years ago by David Triendl <[email protected]>
Conserve memory

e9377173a7a230d051cdb537f3f33f2ecbd83ead authored about 14 years ago by David Triendl <[email protected]>
Only enforce timeout for fetch call, not logging or adding to write_queue

1a1221d98751d8651faff3b81f046e20c22590fe authored about 14 years ago by David Triendl <[email protected]>
Reaper: Retry failed codes only 5 times, then abort

3534c9ffa29539c20f29d60aadb7426ff1faceb8 authored about 14 years ago by David Triendl <[email protected]>
Reaper: Increase fetch queue size per thread

6ca2fd6079f12878a16537dfe6e3ec5cad8be034 authored about 14 years ago by David Triendl <[email protected]>
Reaper: Add timeout for fetch (10 seconds)

When a service causes a timeout, a new instance is created

068b4a566fc5f317b55364a75e66e25303a9f001 authored about 14 years ago by David Triendl <[email protected]>
Isgd: Close socket on wrong headers

8ad9bf94e9167cd7cfeac3b478b0c1ef351116a3 authored about 14 years ago by David Triendl <[email protected]>
Services: Add is.gd

8ab236b924be7d0691975813960c57c3fe29ca49 authored about 14 years ago by David Triendl <[email protected]>
Reaper: Adjust fetch queue size based on number of threads

7f2772e52eca17c6e2ef86d3dbaa2d34a28b3492 authored about 14 years ago by David Triendl <[email protected]>
Reaper: Keep correct order in the fetch queue

15a105f6dd898de3d43386916532d60e63ac37a7 authored about 14 years ago by David Triendl <[email protected]>
Retry on Errno::ECONNRESET

63c44ed2e9a6ba587a03f6445581666e8f787388 authored about 14 years ago by David Triendl <[email protected]>
Reaper: Increase fetch queue size

fd66aaa06578b82c4531ca2a8c3adfe7843b0426 authored about 14 years ago by David Triendl <[email protected]>
Reaper: Improve logging

a557f17f12dd313b5555e8d8b183fe1d3a2d106e authored about 14 years ago by David Triendl <[email protected]>
Services::TinyURL: Increase Hpricot buffer size

51a5737116cb0ee6835ea0895b864217ce61ceff authored about 14 years ago by David Triendl <[email protected]>
Reaper: Don't fail on FetchError

70a6a81d02d477294577b0a9ca946921aa4b488c authored about 14 years ago by David Triendl <[email protected]>
Terminate Reaper when fetching is finished

13021889d5915469e58aae5312015f6a7813f9a6 authored about 14 years ago by David Triendl <[email protected]>
Fix a few bugs in Services::TinyURL

Detect blocked URLs (http://tinyurl.com/dick)
Detect broken URLs (http://tinyurl.com/cs)
Detect ...

ace3659f7c25b01062055437c8871273c6874cef authored about 14 years ago by David Triendl <[email protected]>
Fix two bugs in Reaper

Don't terminate thread on fetch stall
Replace UnknownCodeError with NoRedirectError

1f8df1a50a141a4312a049e8dcf79f142b48d62f authored about 14 years ago by David Triendl <[email protected]>
Fix tinyurl.com (now returns 404 now for non-existant links)

d479db809330bd8d9c435996104ee7d0ed4003be authored about 14 years ago by David Triendl <[email protected]>
Add basic "reaper"

f27fc5dacbbb06cd92cf8c7a533eadd987b7cee4 authored over 14 years ago by David Triendl <[email protected]>
Use class methods for all but fetch

Fetch is probably the only method that ever needs any kind of state

deebbd2d1a9535e14938dea84fad73991faf5253 authored over 14 years ago by David Triendl <[email protected]>
TinyURL::Services::Base class

Add TinyURL::Services::Base class with common methods
Use instance methods, not class methods
Fi...

40ebc3100ff7329ff162dbb5530b3d7e5ff8cd6d authored over 14 years ago by David Triendl <[email protected]>
TinyURL: Add simple advance function

The function should be adaptable for other services as well

cfa586776c16be659720e6e7cf7e5ccff07d434d authored over 14 years ago by David Triendl <[email protected]>
Add TinyURL.com

89291ed9c01517cf5e134f50d33006b2c67b1c07 authored over 14 years ago by David Triendl <[email protected]>
Initial commit

4a9ff5542a5b60aa904bb4243ba34793f832ae53 authored over 14 years ago by David Triendl <[email protected]>