Ecosyste.ms: OpenCollective

An open API service for software projects hosted on Open Collective.

github.com/LemmyNet/lemmy-stats-crawler


https://github.com/LemmyNet/lemmy-stats-crawler

Check that site actor domain matches expected domain (fixes #17) (#18)

19574daec30da0aa6619a5f5a4ef33015568dd0c authored 12 months ago by Nutomic <[email protected]>
Retry requests (#16)

* Retry requests

* Add --timeout, set default timeout 10s

bb7a5c3be8a2c517f6a3e8b897df8f653cc8aeac authored almost 1 year ago by Nutomic <[email protected]>
Fixed 0.19 compatibility (#15)

97dec739346aab80632d8feeb70c5c5388682937 authored about 1 year ago by İsmail Karslı <[email protected]>
Support both Lemmy 0.18 and 0.19 instances (#14)

419d6e7c67951d1210226091b2d09a0536b531c8 authored about 1 year ago by Nutomic <[email protected]>
Upgrading deps. (#13)

* Upgrading deps.

* Running format.

* Running clippy.

ff8a2db505771c9f142ab31626b49de54a756192 authored about 1 year ago by Dessalines <[email protected]>
Dont follow redirects (fixes #8) (#11)

788a3dd6e02fbe153e6d7c6315601ade15637f8c authored over 1 year ago by Nutomic <[email protected]>
Make crawler compatible with Lemmy 0.18 (#10)

* Make crawler compatible with Lemmy 0.18

* fix

89ceb560c5c2698b2971e9b3ba27888e5c4e1eca authored over 1 year ago by Nutomic <[email protected]>
added --help to documentation (#7)

* added --help

* added newline to end of file

ed6a393329b79fd23964231e26cfa00347b8fc83 authored over 1 year ago by Michael Altfield <[email protected]>
Add regex to check that domains are valid (#3)

7192481fa1182810065fdf6aa6671700ee225859 authored over 1 year ago by Nutomic <[email protected]>
Merge pull request #2 from LemmyNet/crawl-without-recursion

Crawl without recursion

4011a9857ee61f9b1e0347c4159e79934edd4a47 authored almost 2 years ago by Nutomic <[email protected]>
Replace deprecated structupt with clap

ee495cc02f9c2a5fbc5b2c3debefa750f65626f8 authored almost 2 years ago by Felix Ableitner <[email protected]>
Add CI config

61b5a7f9cefb98536b1d14b86799e465d811a31e authored almost 2 years ago by Felix Ableitner <[email protected]>
Dont use recursion for crawling

73f0794f4aa3df7d0efc02ae3bb2d7a2ae5108dc authored almost 2 years ago by Felix Ableitner <[email protected]>
Fix HTTP errors due to too many open files (#1)

3b4ab747f1e12dc94ddd6199708b299c7ff6cb27 authored almost 2 years ago by Nutomic <[email protected]>
Upgrade to Lemmy 0.17

1c28b2ad2223ea6ce9930bbbf3e97cacc9137de3 authored almost 2 years ago by Felix Ableitner <[email protected]>
Merge pull request 'Also calculate counts including federated instances' (#12) from federated-counts into main

Reviewed-on: https://git.join-lemmy.org/LemmyNet/lemmy-stats-crawler/pulls/12

bcb852cfeb04d21e43e7afc3ad34009d3c0ecbae authored about 2 years ago by nutomic <[email protected]>
Also calculate counts including federated instances

ea57a16987ee0c383b3d1434ebf8762f9abd84bd authored about 2 years ago by Felix Ableitner <[email protected]>
Upgrade dependencies

1343932b2b6105f7497a6467e1f354120cfed477 authored about 2 years ago by Felix Ableitner <[email protected]>
Remove --verbose param

1d1823e27d46d7ed9c2d0d67aed04883adca5754 authored about 2 years ago by Felix Ableitner <[email protected]>
Log crawl errors

418db7831ff1d3035dfcfe582dd3bb76f52e58d6 authored about 2 years ago by Felix Ableitner <[email protected]>
Use exact version for Lemmy dependency

d11febc7e80ae7ea37bd57f00163d71e4b48a918 authored over 2 years ago by Felix Ableitner <[email protected]>
Handle comma separated params correctly

7ede38f584af65d5b859f58968466675224e9d9c authored over 2 years ago by Felix Ableitner <[email protected]>
Add user agent

21cf61f8477707c5f9703106d61788e05d5fe09b authored over 2 years ago by Felix Ableitner <[email protected]>
Merge pull request 'Recursive, parallel crawl' (#11) from recursive-crawl into main

Reviewed-on: https://yerbamate.ml/LemmyNet/lemmy-stats-crawler/pulls/11

575672cbe38d8b74a789c3b79ce5ce5387c397a8 authored over 2 years ago by nutomic <[email protected]>
Merge pull request 'Use lemmy structs' (#10) from lemmy-structs into main

Reviewed-on: https://yerbamate.ml/LemmyNet/lemmy-stats-crawler/pulls/10

0c3ba08d6c0964d90f632ac7c981857c4aa323a0 authored over 2 years ago by nutomic <[email protected]>
Add stderrlog, structopt for better logging and params

0079e727593c9bf9ca18c840383bf917910f20e5 authored over 2 years ago by Felix Ableitner <[email protected]>
Recursive, parallel crawl

c254e50211f3c5d0e2c09d787dc014f4dbeb35ae authored over 2 years ago by Felix Ableitner <[email protected]>
Use lemmy structs

2e2a4888d07ac6426ea6fb73a2481a6141b68eec authored over 2 years ago by Felix Ableitner <[email protected]>
Merge pull request 'Add min version check to avoid including old instances' (#9) from min-version into main

Reviewed-on: https://yerbamate.ml/LemmyNet/lemmy-stats-crawler/pulls/9

8509c19f50062417a4a1e143914d053210561eb7 authored over 2 years ago by nutomic <[email protected]>
Add min version check to avoid including old instances

574536b151680a4a4ea9e55b9b1d68771b16dc0a authored over 2 years ago by Felix Ableitner <[email protected]>
Convert instance name to lowercase

9546936ddfed8b6d0d7c99b387bf5d594bc3f117 authored almost 3 years ago by Felix Ableitner <[email protected]>
Change default crawl depth to 20

4e954878424889c6f3246ef7a6d7b13e63e55b58 authored almost 3 years ago by Felix Ableitner <[email protected]>
Include `require_application` in crawl output

a1115d474bf0a92887d6939791a7129d25fcbbe7 authored almost 3 years ago by Felix Ableitner <[email protected]>
Add command line param to exclude instances from crawl

f1105549936e5b15b67e1920ff89bee803b2a96f authored over 3 years ago by Felix Ableitner <[email protected]>
Merge pull request 'Sorting by active monthly desc.' (#6) from sort_active_monthly into main

Reviewed-on: https://yerbamate.ml/LemmyNet/lemmy-stats-crawler/pulls/6

fefd7087778ff191878cd2da1fe67a5263094450 authored over 3 years ago by nutomic <[email protected]>
Exclude test instances from crawl

60563fc7d9665591a28c5a8b6f1fbddea73bb921 authored over 3 years ago by Felix Ableitner <[email protected]>
Include description field (if it is under 150 chars)

d92de61d9c00487a073e5a816afdc13e33f8ddd9 authored over 3 years ago by Felix Ableitner <[email protected]>
Add support for Lemmy api v2 and v3

1751035334f879b24107c10e939e291d34f5369e authored over 3 years ago by Felix Ableitner <[email protected]>
Update for API v3

cd04d54b9ab10b766283e7d3d91dd90ce2d4da75 authored over 3 years ago by Felix Ableitner <[email protected]>
Sorting by active monthly desc.

f56a7dde4fba2dcd0f31c5b739cd0f90175dd709 authored almost 4 years ago by Dessalines <[email protected]>
Remove duplicate code, clippy fixes

0fcf8fe5234546873d39ca0289711acd310221d4 authored almost 4 years ago by Felix Ableitner <[email protected]>
Include number of failed instances

95107b6a39edb34930b79184e8c36a3754e06385 authored almost 4 years ago by Felix Ableitner <[email protected]>
Pretty print

5d5a9b863d4248c3dcd994f50a5ec1db624f1089 authored almost 4 years ago by Felix Ableitner <[email protected]>
Add support for command line parameters

9621b91f7c1b7e00d570c33e931624bcc79266c4 authored almost 4 years ago by Felix Ableitner <[email protected]>
Add max depth parameter for crawl

f01e077020702ed4aea93bd415235ac977aa37da authored almost 4 years ago by Felix Ableitner <[email protected]>
Dont redo crawls for unreachable instances

5a09fa46c4627b0b7029e1023e58a7041123ef7d authored almost 4 years ago by Felix Ableitner <[email protected]>
Move crawl code into separate file

48a164de5abcc643db902e9251582589aa7471ed authored almost 4 years ago by Felix Ableitner <[email protected]>
Simplify crawl by using queue

ce909d5e72f2cbf60b1f788c956b7bd9e7884939 authored almost 4 years ago by Felix Ableitner <[email protected]>
Minor reduction in dependencies

0690824e9e37a45ad80d3915b1de478e2b58a04d authored almost 4 years ago by Felix Ableitner <[email protected]>
Remove instance filtering/sorting

f3876b47459a4b67030589a1e7e8924ec7ce0084 authored almost 4 years ago by Felix Ableitner <[email protected]>
Simplified crawl loop, improved output

71cdf40baecafe88dc03278ab2ec1a0423d3e4a2 authored almost 4 years ago by Felix Ableitner <[email protected]>
Add total_instances count, use consts

33580c52ea467dbf65000f4b089ae35323ae8901 authored almost 4 years ago by Felix Ableitner <[email protected]>
Improved crawling code, allow null icon

4426b0edf92c3c85df2bd913e62d2da7ba1d74bc authored almost 4 years ago by Felix Ableitner <[email protected]>
Make requests to same instance in parallel

342fe6ded85d0d162e843e2123d3598b60a651fe authored almost 4 years ago by Felix Ableitner <[email protected]>
Improve output, refactor code, add timeout

27c23bd5d83a057a4c48fc917555112933f920ab authored almost 4 years ago by Felix Ableitner <[email protected]>
Add instance name and icon

c45771d9e767b75d96ffec2bd384967647ceb083 authored almost 4 years ago by Felix Ableitner <[email protected]>
Output instance info as json

9cbed8f2a99cdc0b4ace83e73498b363c089c6d8 authored almost 4 years ago by Felix Ableitner <[email protected]>
Add license

cb4e6e221ad51e4e993baf549cb34852e1581248 authored almost 4 years ago by Felix Ableitner <[email protected]>
Crawl for number of online and total users

45ec994271aacad1868e008cd7a606d6ea75b60a authored almost 4 years ago by Felix Ableitner <[email protected]>
Fetch basic data from lemmy.ml

21be5c58ee072316c9db9ea6a7ac070666762cfa authored almost 4 years ago by Felix Ableitner <[email protected]>
Initial commit

5f871dcc964b04ca73156ec4cbb0c8a6de8696f4 authored almost 4 years ago by Felix Ableitner <[email protected]>