Ecosyste.ms: OpenCollective

An open API service for software projects hosted on Open Collective.

github.com/ooni/probe-engine

Semi-automatic export of https://github.com/ooni/probe-cli internals
https://github.com/ooni/probe-engine

doh: use same user-agent used for measurements (#731)

No need to announce us as being OONI.

Part of https://github.com/ooni/probe-engine/issues/716

36960683f162b52300b3c94d05b0217ecc7859c9 authored over 4 years ago by Simone Basso <[email protected]>
Merge pull request #730 from ooni/issue/629

experiment/tor: allow for testing private bridges

508917a872772c08c906bc780de5844f420f0329 authored over 4 years ago by Simone Basso <[email protected]>
experiment/tor: add integration test to ensure scrubbing works

We pretend the orchestra API to return to us a entry that comes from
bridge DB (although that en...

59bbb7a5516f570a28df5f4151b56f5455fbb50a authored over 4 years ago by Simone Basso <[email protected]>
probeservices/tor: add country_code to query string

```
> GET https://ps1.ooni.io/api/v1/test-list/tor-targets?country_code=IT
```

Part of https://...

8cc6f6e65a1efe427f57a2e741a0198725372df8 authored over 4 years ago by Simone Basso <[email protected]>
Merge pull request #729 from ooni/issue/728

probeservices: use ps{1,2}.ooni.io

bcffd18482fbc2214cfccaafe4d297eb65949908 authored over 4 years ago by Simone Basso <[email protected]>
all: improve tests using probe services

1. do not use confusing names that may be mistakenly believed
to be real existing hosts (closes ...

eb63d98877de2a99e64c2b0fa9b17053cb09ace1 authored over 4 years ago by Simone Basso <[email protected]>
probeservices/client.go: use ps{1,2}.ooni.io

This applies 4f15948257f981b321d4e800ada2e792bf4b30a7 to master.

Part of https://github.com/oon...

e35e629e8b484bb1fc6d45186adfaacc93c2abbc authored over 4 years ago by Simone Basso <[email protected]>
Update to ooni/probe-assets@20200619115947 (#722)

Thanks @FedericoCeratto!

14cfefbce4165f6af8b1cc48a4eb2fb30c6996af authored over 4 years ago by Simone Basso <[email protected]>
tor: scrub private bridge addresses (#720)

Scrubbing is done in a rather aggressive way where we scrub any
endpoint looking string from th...

5d1e643754114c8ad5b13b1b2b7c5a06fa803ff0 authored over 4 years ago by Simone Basso <[email protected]>
session: ensure we have complete code coverage (#718)

It was reduced a bit when working on https://github.com/ooni/probe-engine/issues/651

404e7d36ddef4a10f9fdd5618de96bf0c2086b84 authored over 4 years ago by Simone Basso <[email protected]>
probeservices: improve tests, increase coverage (#717)

Part of https://github.com/ooni/probe-engine/issues/651

Addresses the TODO mentioned in 4c0aa...

bd1fda07be2f12f0c03d5c1d1338357c658e4dce authored over 4 years ago by Simone Basso <[email protected]>
codereview(probeservices): simple changes and TODO annotations (#715)

Make sure everything easy I think we should do to improve the code
quality is either done right...

4c0aaa591c335209836efc196c23c7310b87c812 authored over 4 years ago by Simone Basso <[email protected]>
refactor(probeservices): improve tests implementation (#714)

* refactor(probeservices): StateFile does not need to be a pointer

The StateFile is like:

...

d9eca977c46af197a396c6ea24451cdc5d4e67ed authored over 4 years ago by Simone Basso <[email protected]>
refactor(probeservices): StateFile does not need to be a pointer (#713)

The StateFile is like:

```Go
// StateFile is the orchestra state file. It is backed by
// a...

35ad6d740b47fa86e93f2d6549cc1e0acc70d4a2 authored over 4 years ago by Simone Basso <[email protected]>
refactor(probeservices/collector.go): better names, error variables (#712)

1. use names from which it's more clear the file stuff belongs to

2. use error variables rath...

d0c6f9faf8408d04d918f29cd560477bb1bb82d4 authored over 4 years ago by Simone Basso <[email protected]>
refactor(probeservices): merge orchestra.go and client.go (#711)

Part of https://github.com/ooni/probe-engine/issues/651

ef486685412dde9b46a4820a0d688892aad34b80 authored over 4 years ago by Simone Basso <[email protected]>
refactor(probeservices): move randomPassword in register.go (#710)

Part of https://github.com/ooni/probe-engine/issues/651

6a58ce21652ea1bd2fc57f423c2d580f58062ef8 authored over 4 years ago by Simone Basso <[email protected]>
refactor(probeservices): move testing fixtures in subpackage (#709)

Part of https://github.com/ooni/probe-engine/issues/651

486b869c318382f0ce613656d33cc0caba4bf966 authored over 4 years ago by Simone Basso <[email protected]>
httpx: more intuitive names for jsonapi (#708)

We used to use CRUD names. Naming the method explicitly is more intuitive.

Part of https://gi...

01c259147f61c51568957321a50e30341822683d authored over 4 years ago by Simone Basso <[email protected]>
probeservices: tor, psiphon: use more compact code pattern (#707)

This code pattern also happens to be less error prone.

Cleanup as part of https://github.com/...

a422ac96f30036034ed01bcc6efa31108954cfb5 authored over 4 years ago by Simone Basso <[email protected]>
probeservices: remove bouncer GetCollector API (#706)

We have now migrated to probeservices. The probeservices implement the
bouncer API, therefore, ...

3c72a347d051273759769abee91a470d8207dea3 authored over 4 years ago by Simone Basso <[email protected]>
NewOrchestraClient: use already selected probe service (#704)

This allows us to use, in principle, also cloud fronted probe services
when there is need to do...

c88c3de422dd9bbb0e31ce569ca4432a609d1ea5 authored over 4 years ago by Simone Basso <[email protected]>
probeservices: sketch out new API to access test lists (#702)

I want to replace the old test-list/urls API with this new API. The main
reason for doing so is...

df892c1cc57601e1c77d19ea1791968a0293ede3 authored over 4 years ago by Simone Basso <[email protected]>
Merge internal/orchestra into probeservices (#701)

This is half of our journey. We have now the orchestra API inside the
probeservices API. But th...

4e3d3db6c121f9a7465375a1b867d9921feb356c authored over 4 years ago by Simone Basso <[email protected]>
internal/orchestra: rewrite to have same constructor as probeservices (#700)

This should be one of the last steps. The orchestra client and the probeservices
client now hav...

45552709b5bdba452c62fc34d6dc89f07c7340e4 authored over 4 years ago by Simone Basso <[email protected]>
orchestra: simplify Client by embedding httpx.Client (#699)

Part of https://github.com/ooni/probe-engine/issues/651

6d595a487ea7ab545ae2073474a1042bdb7e0024 authored over 4 years ago by Simone Basso <[email protected]>
orchestra: remove the UPDATE operation (#698)

We're not going to use orchestra for sending updates regarding the
status of the clients. Rathe...

11bbf51c58f0c72d6d78da5fa98b93269a41f7f6 authored over 4 years ago by Simone Basso <[email protected]>
orchestra: refactor code to remove redundancy (#697)

After I moved everything into the same package, there is lots of
redundancy that we can remove....

9d69547b25b95d7387d5c86c712487ef89fd3134 authored over 4 years ago by Simone Basso <[email protected]>
orchestra: remove distinction between registry and orchestra base URLs (#696)

We're using probe-services now, so there's no need to distinguish.

Part of https://github.com...

0845cf9fe76bebebec8ae111897d0955db997504 authored over 4 years ago by Simone Basso <[email protected]>
Merge pull request #695 from ooni/issue/651

httpx, resources: improve testing and code quality

228cecec4d4b601b6778a94cc6e0c7007f5fca87 authored over 4 years ago by Simone Basso <[email protected]>
resources: make sure tests are more specific

Spotted while working on 0aa9a5e1301256d99e1f1d419ce618a7b0d83abd.

Part of https://github.com/o...

6b44870d40b086941d41291eae78b70c8f7cbe37 authored over 4 years ago by Simone Basso <[email protected]>
httpx: make sure tests are more specific

Spotted while working on 0aa9a5e1301256d99e1f1d419ce618a7b0d83abd.

Part of https://github.com/o...

19c9f5e00f6679e36db673a5bf6af475847cab53 authored over 4 years ago by Simone Basso <[email protected]>
Stop taking the address of httpx.Client

We don't need that. Spotted while working on
0aa9a5e1301256d99e1f1d419ce618a7b0d83abd.

Part of ...

768c4845d0d028574c67ca7bc2220ed816cb1f84 authored over 4 years ago by Simone Basso <[email protected]>
Merge internal/fetch with internal/httpx (#694)

The API needs to change slightly. Rather than fetching a full URL, we
split it into a base URL ...

0aa9a5e1301256d99e1f1d419ce618a7b0d83abd authored over 4 years ago by Simone Basso <[email protected]>
rename: internal/jsonapi => internal/httpx (#693)

To merge orchestra and probeservices, I also need to merge the two
supporting packages: jsonapi...

9b172f5b9d3e09dafdc022ce8a0207e86a688b0a authored over 4 years ago by Simone Basso <[email protected]>
Merge pull request #692 from ooni/issue/651

orchestra: refactor to simplify using probe-services

2e6eeb662a02bd0a1b8b9a750017756f4cf7d1e5 authored over 4 years ago by Simone Basso <[email protected]>
orchestra: merge orchestra/testorchestra into orchestra

Will simplify further refactoring.

Objective is https://github.com/ooni/probe-engine/issues/651

66466753f9e909ef649449147e67a6caf72c20a8 authored over 4 years ago by Simone Basso <[email protected]>
orchestra: merge orchestra/metadata into orchestra

Will simplify further refactoring.

Objective is https://github.com/ooni/probe-engine/issues/651

b93585d0c09088ec86e3fd025e9ded7acb895366 authored over 4 years ago by Simone Basso <[email protected]>
orchestra: merge orchestra/login into orchestra

Will simplify further refactoring.

Objective is https://github.com/ooni/probe-engine/issues/651.

957596623b697bc1aae197edbbe460aaed20083d authored over 4 years ago by Simone Basso <[email protected]>
orchestra: merge orchestra/statefile into orchestra

Will simplify further refactoring.

Objective is https://github.com/ooni/probe-engine/issues/651.

b8d1237b148f1b67360f86b10e70af975887fc3f authored over 4 years ago by Simone Basso <[email protected]>
orchestra: merge orchestra/testlists/urls into orchestra

Will simplify further refactoring.

Objective is https://github.com/ooni/probe-engine/issues/651.

9f112b4293ff0f6570ff8cda57ee7bbba6f9f806 authored over 4 years ago by Simone Basso <[email protected]>
orchestra: merge orchestra/testlists/tor into orchestra

Will simplify further refactoring.

Objective is https://github.com/ooni/probe-engine/issues/651.

2e1bd5a7a1a7d9e30081e825dfca860318ffc1ff authored over 4 years ago by Simone Basso <[email protected]>
orchestra: merge orchestra/testlists/psiphon into orchestra

Will simplify further refactoring.

Objective is https://github.com/ooni/probe-engine/issues/651.

fdb861b64650e6a4999da4f23dad4d014d667c82 authored over 4 years ago by Simone Basso <[email protected]>
orchestra: merge orchestra/register into orchestra

Will simplify further refactoring.

Objective is https://github.com/ooni/probe-engine/issues/651.

538a4a65436c936403a7aeecb2bfd8d445d4ff62 authored over 4 years ago by Simone Basso <[email protected]>
orchestra: merge orchestra/update into orchestra

Will simplify further refactoring.

Objective is https://github.com/ooni/probe-engine/issues/651.

2d99e0f617d548e044e30c1224085abe0e3dd1e3 authored over 4 years ago by Simone Basso <[email protected]>
orchestra: make tests external

This will simplify further refactoring.

The end goal is https://github.com/ooni/probe-engine/is...

a3a82b920f0c3986401635c0a409da0c4d112293 authored over 4 years ago by Simone Basso <[email protected]>
probeservices: pass session's kvstore to client

We want the probeservices client to be constructed with a reference
to the session's key-value s...

a2c45a59e3a69c82ccd3f688a15f57513e02086b authored over 4 years ago by Simone Basso <[email protected]>
Merge pull request #691 from ooni/issue/656

data quality: make data format more actionable

1. set dns_cache to null when empty

2. reco...

be862923919da23d6c3da2623c168d8285eee37d authored over 4 years ago by Simone Basso <[email protected]>
libminiooni: record options we're using

While in general it does not make sense to save options and we don't use
much options, they are ...

a70ec53eebf70a7de5b22ce5400bf34c9ece6a7d authored over 4 years ago by Simone Basso <[email protected]>
urlgetter: set DNSCache to nil when empty

Much better from the point of view of data analysis than having a
single entry which is actually...

19ea3d37b13f051c220a4fd358d53a1cede5b5e6 authored over 4 years ago by Simone Basso <[email protected]>
Merge pull request #688 from ooni/release/0.13.0

Release 0.13.0

586ba16e8b9fc965c1d33e8d520970872c17f045 authored over 4 years ago by Simone Basso <[email protected]>
Update to ooni/probe-assets@20200610133507

b18703cdb8985333c01d7d5e4040191ce4cd1b5c authored over 4 years ago by Simone Basso <[email protected]>
version/version.go: start to advertise ourself as v0.13.0

4e2faaf72dcde89f752f7198e6eba08205627a0b authored over 4 years ago by Simone Basso <[email protected]>
Update direct and indirect dependencies

cd6dbccf21ccc437fe4151f9e4f677fd36bb6561 authored over 4 years ago by Simone Basso <[email protected]>
session.go: create subdirectory with config.TempDir (#687)

This diff changes the way in which session.go manages config.TempDir. Before this diff, the call...

30339a93cc48f428a50182fdc322d5162b953442 authored over 4 years ago by Simone Basso <[email protected]>
telegram: rewrite using experiment/urlgetter (#683)

Part of https://github.com/ooni/probe-engine/issues/646

Passes all checks from https://github...

0a0028e2eaca06b0e76bf2f35154398cfba94e5c authored over 4 years ago by Simone Basso <[email protected]>
netx/errorx: wrap context.Canceled (#682)

Like for MK errors, use the same naming of the C++11 library.

This error will happen when the...

bb83b5d6238dc2a6eb72e9e6d15bd7d507cd909d authored over 4 years ago by Simone Basso <[email protected]>
urlgetter.TestKeys: add extra fields to simplify analysis (#681)

We often need to check the status code of the last response in the chain
and/or to see the body...

07ac3a971df53884d2dc7b68d45ab3bfb64028c1 authored over 4 years ago by Simone Basso <[email protected]>
urlgetter: expose the operation that failed (#680)

This is very useful to quickly understand what actually went wrong. It is
a facet of the next d...

746812da74ffec4d62d014f0f834b1881d50e16a authored over 4 years ago by Simone Basso <[email protected]>
netx/archival: extract the operation that failed (#679)

This will be useful to more easily classify the cause of blocking.

This is part of https://gi...

e1297dc9d17a93734442f29f14b996b8273bc527 authored over 4 years ago by Simone Basso <[email protected]>
netx: use constants to represent all operations (#678)

This is less error prone. I consider this yak shaving to make the
code more robust before conti...

9ead0e178a8b6d17bff0ebf9654626b3cbe14163 authored over 4 years ago by Simone Basso <[email protected]>
urlgetter: introduce the Multi urlgetter (#677)

With the Multi urlgetter you can perform operations in parallel with a
specific degree of paral...

d6316d6de1a5cb76be23b588561122b4837e7e64 authored over 4 years ago by Simone Basso <[email protected]>
urlgetter: allow to configure the method (#676)

We need this to reimplement the telegram experiment in terms of
the urlgetter experiment, to si...

a3de37714ae8c20c4f2daa0d219a2b339c187c36 authored over 4 years ago by Simone Basso <[email protected]>
urlgetter: add tcp_connect to test_keys (#675)

This was planned inside https://github.com/ooni/probe-engine/issues/656.

It actually also hel...

a0fff21c804277cc95460cbbcc8d7760fcd3d53c authored over 4 years ago by Simone Basso <[email protected]>
stunreachability.go: inform the user about what we're doing (#674)

Related to https://github.com/ooni/probe/issues/1185

ead7dcf4a7fe56b7817f9fad501110d98e6a080c authored over 4 years ago by Simone Basso <[email protected]>
Improve Android and iOS builds (#667)

1. android: make sure JCenter accepts our POM (see https://github.com/ooni/probe-engine/issues/6...

f87ba1ea9527f7c715c3884bc6fe64db4e8f4fbe authored over 4 years ago by Simone Basso <[email protected]>
Merge pull request #665 from ooni/fix/proxy

Fix --proxy and use more precise user agent

eae31150639aa517a43f170026d5ae5699a3a9b9 authored over 4 years ago by Simone Basso <[email protected]>
session.go: include engine version in User-Agent

Part of https://github.com/ooni/probe/issues/886.

5bc7a7f416aa387ba60aaad1e48afaad0cc3a5e4 authored over 4 years ago by Simone Basso <[email protected]>
fix(session.go): make sure `--proxy` still works

It was quite likely broken recently when we extended the tunnel
to support `tor`. Since this is ...

c040bea17d68f93dad4024d67a779b5b1d6fc277 authored over 4 years ago by Simone Basso <[email protected]>
libminiooni: do not use `-P` for `--proxy`

This option conflicts with ooniprobe v2.x, which uses `-P` to indicate
the preferred backedn, so...

359d74e67a1cd68c4efd8691977758ddf7fef688 authored over 4 years ago by Simone Basso <[email protected]>
Write stun_reachability experiment prototype (#630)

Closes https://github.com/ooni/probe-engine/issues/626

To finish off this PR, needed to make ...

ac013a7a4130dfa21accd38c2bb8e6285f9a59af authored over 4 years ago by Simone Basso <[email protected]>
session: select the fastest probe service (#647)

Part of https://github.com/ooni/probe-engine/issues/407

Includes fixes for a flaky integratio...

55f5e7748138db59f41fc7663f730733ac2a3e67 authored over 4 years ago by Simone Basso <[email protected]>
Use ps.ooni.io wherever it makes sense to do so (#652)

Prodded by @hellais. Will give us better performance. Yet, this is just
half of the job. I need...

b746cbfb2d4edfc39a07a25d64706cf670095f0d authored over 4 years ago by Simone Basso <[email protected]>
fix(experiment.go): serialize assets version in 32bit compatible way (#649)

Build otherwise broken. See: https://github.com/ooni/probe-engine/runs/720733159.

e768161f91db21f6ce1df7dfa8174c3aaa4bba18 authored over 4 years ago by Simone Basso <[email protected]>
httptransport.go: remove wrong comment

We actually want to choose what to save. So always using the same saver seems tricky.

f3594e5a134b0ddffae82fbf376efe16e9de8403 authored over 4 years ago by Simone Basso <[email protected]>
Merge pull request #648 from ooni/release/0.12.0

Release v0.12.0

513a3a9c43b9187ebc26ce9da6987ae8aedc1a30 authored over 4 years ago by Simone Basso <[email protected]>
fix(archival_test.go): change test since asn.mmdb changed

We're now using a asn.mmdb with RIPE names. Those names are
different from the previous names. S...

87e7cae2ced008c09f12b59df9afe3bb344b80e2 authored over 4 years ago by Simone Basso <[email protected]>
internal/httpheader: update user-agent we use

bece14bc1cdc15a0fac5e1518e9c1aad94ec0d64 authored over 4 years ago by Simone Basso <[email protected]>
ndt7: remove last dependencies from m-lab/ndt7-client-go

1eba4b8e5b2fb5d4c1e12d994a43fab369b2cb41 authored over 4 years ago by Simone Basso <[email protected]>
Update to ooni/probe-assets@20200529153246

a4f394af0542bc1689965e222441d5b7c0d64f97 authored over 4 years ago by Simone Basso <[email protected]>
experiment.go: add assets version to annotations

2572ae20fe3a919ea2a47ea945d3b315e8a4da3e authored over 4 years ago by Simone Basso <[email protected]>
version/version.go: we're now at 0.12.0

8548e7a93b4eca1f7d47c3ef7694d129ffcad8ec authored over 4 years ago by Simone Basso <[email protected]>
Update all direct and indirect dependencies

9a278e1ab1c33531d48dcbfc6bd43a718536278e authored over 4 years ago by Simone Basso <[email protected]>
routine-sprint-releases.md: clarify a bunch of actions

b6df09c51dfd8d2a9fbc9e0ad52d271240a2ac4c authored over 4 years ago by Simone Basso <[email protected]>
Merge pull request #644 from ooni/issue/576

tls/timeout: use deadline rather than context

fee767280c4ad1250e2cf2a2891abe542419becb authored over 4 years ago by Simone Basso <[email protected]>
selfcensor: return i/o timeout when timeout is requested

This should unbreak the code afterf348116d47bf58ac6a0e36d7593ced08fdc26fd2
changed the way in wh...

367fcf7d936bee95704704a292f9d3ef67e588d8 authored over 4 years ago by Simone Basso <[email protected]>
tls/timeout: use deadline rather than context

This should lead to a more understandable sequence of low level events
where we don't see anymor...

f348116d47bf58ac6a0e36d7593ced08fdc26fd2 authored over 4 years ago by Simone Basso <[email protected]>
Make sure we declare the tunnel data format when needed (#641)

Part of https://github.com/ooni/probe-engine/issues/533

While there, bring coverage in `netx/...

a75f7aad01438e04b8f042ff9e93109471c5d384 authored over 4 years ago by Simone Basso <[email protected]>
Build and publish iOS framework (#497)

Closes https://github.com/ooni/probe-engine/issues/498

d4291daab2a72f90820718d12fc9eb38d777cdd8 authored over 4 years ago by Simone Basso <[email protected]>
fix(task_test.go): fix test logic to avoid races (#640)

It still beats me how it's possible that the test takes more than 15
seconds to complete. But m...

cd0f99ee2688137e07768763e16476e6add6fdd4 authored over 4 years ago by Simone Basso <[email protected]>
Code to investigate frequent build breakages (#639)

65dca1935807047b047a71c7eb858f97480262e6 authored over 4 years ago by Simone Basso <[email protected]>
Merge pull request #638 from ooni/issue/621

Finish migrating to OONI Probe Services

66f9096cbb2781b8c6843afb5c20d9c6de92e05f authored over 4 years ago by Simone Basso <[email protected]>
probeservices: use ps.ooni.io rather than bouncer.ooni.io

With this change, we have really embraced the probe services.

Part of https://github.com/ooni/p...

3b72adc5e3ba8c1bc884736e6701498554119a4d authored over 4 years ago by Simone Basso <[email protected]>
oonimkall: allow to set probe-services URL

This also ensures we're testing the whole package. Before being able to
change the probe-service...

6c6fecd0cf728cbde3934a4099d37115af7ff68a authored over 4 years ago by Simone Basso <[email protected]>
engine: replace bouncer and collector with probe services (#637)

The distinction between bouncer and collector has vanished! Now we have
probe services that we ...

de30fc99121ed559539fc469ff5e925cffce7b37 authored over 4 years ago by Simone Basso <[email protected]>
Merge pull request #636 from ooni/issue/621

* oonimkall: remove support for {Bouncer,Collector}BaseURL

* fix(oonimkall/task_test.go): fur...

ca6902208d4ce08222eaa4fe4a23e480cd1427be authored over 4 years ago by Simone Basso <[email protected]>
fix(oonimkall/task_test.go): further increase runtime for docker builds

This test has started to take slightly more than 7 seconds now:

https://github.com/ooni/probe-e...

dd9730e796d5a705a28a6406984f85b900ab6155 authored over 4 years ago by Simone Basso <[email protected]>
oonimkall: remove support for {Bouncer,Collector}BaseURL

After this change, we're unable to test that the code WAIs when we
cannot discover backends and ...

87fc000ba933654fb0d3f78e7fbb31e4a18e1ade authored over 4 years ago by Simone Basso <[email protected]>
libminiooni: stop using --{bouncer,collector}-base-url (#635)

Part of https://github.com/ooni/probe-engine/issues/621

871691d9bb027a0ba0be968d41882cc602e82790 authored over 4 years ago by Simone Basso <[email protected]>
torx: save tor logs to a file in tempdir (#634)

The location of the file can always be deduced given a session. This means
we can read the logs...

f240976ebd47042a502b5c8cafbc2fd10e7047fd authored over 4 years ago by Simone Basso <[email protected]>