Ecosyste.ms: OpenCollective

An open API service for software projects hosted on Open Collective.

github.com/ArchiveTeam/vine-multi-ingress

Importing multiple sources of Vine videos and user profiles
https://github.com/ArchiveTeam/vine-multi-ingress

Switch to vine.inkdroid.org.

e5b1ec4efba95de29bf2c5d49d082c317f479d0c authored about 8 years ago by David Yip <[email protected]>
More waits, deeper queues.

d6a603d95f57d515852106e4ea8bf44a1d6887b0 authored about 8 years ago by David Yip <[email protected]>
Split out a tool to ingest a file from inkdroid.

32212b439f03314ed368850668690ea53f90c4ba authored about 8 years ago by David Yip <[email protected]>
inkdroid/run: Don't get fooled by e.g. .txt.gz.

bb49881e3b14f18f7f6c3ce51fa08b933e4d6634 authored about 8 years ago by David Yip <[email protected]>
Import Vines from the @archiveteam Twitter listener.

Also split out code that was useful for building said importer.

f649683af98aac768143c087b429f82e0c1e2d3c authored about 8 years ago by David Yip <[email protected]>
Concurrent PUT.

106ca9ff3ce4bc32f1ad99964ff1888addbff0c0 authored about 8 years ago by David Yip <[email protected]>
Extract vines from tweets.

c7c993a5ccbd3cc05d83402b614d2cd7921d28bd authored about 8 years ago by David Yip <[email protected]>
Allow database log level to be configurable.

be48ad89e9a083280c8fa7c8de9595cbbb6ab6c4 authored about 8 years ago by David Yip <[email protected]>
Add missing counter.decrements.

96b89915778b81fa17bc98dfec0140a72893fd7b authored about 8 years ago by David Yip <[email protected]>
Log PUTs for tweet ID documents.

b852f66ac20269dfbe4350cb6ebcfa9208b1762c authored about 8 years ago by David Yip <[email protected]>
Skip over invalid vine URLs.

In http://inkdroid.org/vine/20161105105755.txt we have a line that looks
like

7948527852589752...

e2df8983a66bd6ccd1455abe2a38798496a149a5 authored about 8 years ago by David Yip <[email protected]>
Protect against incomplete lines in source files.

This shouldn't happen anymore, but just in case.

e7123b2fe7e950ad124e9ded5b6a2fe7c1c6db3f authored about 8 years ago by David Yip <[email protected]>
Add Google Form importer.

5886d5ebee8e0f9329f92f8f5c02111609a5323e authored about 8 years ago by David Yip <[email protected]>
Label tweet documents with the tweet ID and Vine video ID.

A tweet can link to multiple Vines.

2be3ab7fa2c5b6640d26c2985e27ad4b90ea5500 authored about 8 years ago by David Yip <[email protected]>
Use tweet ID as tweet document ID.

This saves us some space.

f213dc1d906635c9d258517f97873ddba4bfb77e authored about 8 years ago by David Yip <[email protected]>
Store tweets as separate documents.

The number of tweets referencing some Vines is too large for the
GET-modify-PUT cycle to be effi...

cbe05555a4a95e79d7b5fa93f7e20ca6d7c00854 authored about 8 years ago by David Yip <[email protected]>
Remove stray global.

fc851f6615b21a042c7f84f21ea29b625f1db262 authored about 8 years ago by David Yip <[email protected]>
Import Vine files from inkdroid.org. Closes #1.

c86d6cc9295de02c8d493991d8d9fd5074c5ee51 authored about 8 years ago by David Yip <[email protected]>
Don't require full URLs in a video document.

The tweet URLs are just for source tracking; we don't need to eagerly
resolve IDs to URLs. (Som...

5a953c73101becc73ecd225aba289c37f658b9d9 authored about 8 years ago by David Yip <[email protected]>
Expect a conf.sh in the source directory, but don't track it.

This'll hold DB_URL, DB_USERNAME, and DB_PASSWORD environment variables,
which will be sourced f...

bbc9bbe45538ab57dea518e713ac88e544cc40e8 authored about 8 years ago by David Yip <[email protected]>
Allow videos and users to live in the same database.

Things are easier this way.

f556d3e4c6b4baf52369720c7ee223bea3048275 authored about 8 years ago by David Yip <[email protected]>
Add useful libraries.

ff65e4d7e188c78df272d6a599935b6c3c7a7001 authored about 8 years ago by David Yip <[email protected]>
Add first cut at document format. #4.

441822eba033e3825edcf5dcf46971b4a51af821 authored about 8 years ago by David Yip <[email protected]>
Describe the ingress target.

8403c69d97bbd4c098516b7c3a2020588838e914 authored about 8 years ago by David Yip <[email protected]>
Initial commit

6ff9f81a6211a2d8f36edd35682ca7d010967237 authored about 8 years ago by David Yip <[email protected]>