Ecosyste.ms: OpenCollective

An open API service for software projects hosted on Open Collective.

github.com/ooni/probe-engine

Semi-automatic export of https://github.com/ooni/probe-cli internals
https://github.com/ooni/probe-engine

urlgetter: allow to choose specific TLS version (#633)

Closes https://github.com/ooni/probe-engine/issues/624

d92bb1a6c2a4e2a79461a1f751cb170250ca075e authored over 4 years ago by Simone Basso <[email protected]>
urlgetter: set allow-all cookiejar for HTTP requests (#632)

Closes https://github.com/ooni/probe-engine/issues/572

f430f9076da26470599d2fac02f7b11beb4a48cf authored over 4 years ago by Simone Basso <[email protected]>
model/model.go: use 127.0.0.2 for unknown resolver IP (#628)

This has been recommended by @hellais some time ago. With the
merge of this diff, we can close ...

f844fc4c60528a85a35e1530e61c739879797c9f authored over 4 years ago by Simone Basso <[email protected]>
Merge pull request #617 from ooni/issue/331

Save ASN and ASOrgName of each DNS lookup

8dd946a956f2bd5b7de8f12775721d6435ab91d5 authored over 4 years ago by Simone Basso <[email protected]>
fix(libooniffi): don't write testdata outside root directory

Spotted while running tests for the previous commit, which in turn is
being worked on as part of...

f49fd1ae53d91bf1f7162e5003744bda2875d0d3 authored over 4 years ago by Simone Basso <[email protected]>
urlgetter: annotate DNS resolution with AS number and org

Closes https://github.com/ooni/probe-engine/issues/331

This seems to be useful to continue eval...

63b18b5a30069d3734a6e7b04692f256398a9ef2 authored over 4 years ago by Simone Basso <[email protected]>
fix(netx/archival): make sure we emit DNS errors

Spotted while working on https://github.com/ooni/probe-engine/issues/584.

74ab5b0a96b38f10382bee0cce570e435a3bb79a authored over 4 years ago by Simone Basso <[email protected]>
netx: implement the tcp:// and dot:// schemes for DNS (#616)

Needed to do https://github.com/ooni/probe-engine/issues/581

Usage:

```bash
./miniooni -O...

e3c920170e729eac58f9e79d95b4c5ee392372d0 authored over 4 years ago by Simone Basso <[email protected]>
jsonapi: explain why every request honours the proxy, if any (#612)

I felt this needed to be clarified more explicitly.

Part of https://github.com/ooni/probe/iss...

b4b73d09ac95f6571d579378bae636d5860faa28 authored over 4 years ago by Simone Basso <[email protected]>
Merge pull request #611 from ooni/issue/887

More confidence that we can use tor to contact probe services

4ebe26d4a79beee98593d74031d82960747cfd9a authored over 4 years ago by Simone Basso <[email protected]>
session: tighten tunnel rules

For now, we cannot avoid the issue that a session only manages
a single tunnel or proxy, which i...

3b3f811f6245d6b22f58907357c07816eb953452 authored over 4 years ago by Simone Basso <[email protected]>
libminiooni: allow starting a tunnel from command line

The use case is that we're blocked and then we start tor with, say, the
meek pluggable transport...

e8024025d17e91efe682141a7a2ec8adb2083dab authored over 4 years ago by Simone Basso <[email protected]>
fix(libminiooni): let psiphon work without arguments

I noticed that recently I broke running psiphon from miniooni
because I don't set empty argument...

3ecaafb49bc98a27d658225f54096f987718b2aa authored over 4 years ago by Simone Basso <[email protected]>
probeservices: sort http, then cloudfront, then onion (#609)

Closes https://github.com/ooni/probe-engine/issues/608

7f70f9c22634f21a612605fe75656721d4d0e44e authored over 4 years ago by Simone Basso <[email protected]>
session: use cloudflare doh by default, system as fallback (#607)

* session: use cloudflare doh by default, system as fallback

This is useful at least as far a...

6d216a2194e0a61e14cfbfe73e7818013e4a738b authored over 4 years ago by Simone Basso <[email protected]>
probeservices: implement fallback to cloudfronting (#606)

Part of https://github.com/ooni/probe/issues/887.

Close https://github.com/ooni/probe-engine/...

20bf8406c48e6a81c2253792612c833763e8951c authored over 4 years ago by Simone Basso <[email protected]>
[API CHANGE] session: significantly simplify initialization (#605)

This diff significantly simplifies the way in which the session
is initialized. Now all options...

f065f75bc49552bf648eb2cf29d17536840e3754 authored over 4 years ago by Simone Basso <[email protected]>
netx: factor DNSClient out of urlgetter (#604)

This allows me to use this client also in other places and specifically
I am planning on using ...

beaec99795482512db17d90b756f02a5ce48cd47 authored over 4 years ago by Simone Basso <[email protected]>
Introduce simple self censorship mechanism (#603)

This diff introduces a simple self censorship mechanism. The machinery is always there but it is...

73b82b1eba267a387335869674d5c4eb84cce25a authored over 4 years ago by Simone Basso <[email protected]>
Merge pull request #602 from ooni/probe/887

jsonapi, probeservices: refactoring before detecting blocking

f888c5cbe4da565d12466f261c180f6e90d2bfe3 authored over 4 years ago by Simone Basso <[email protected]>
probeservices: make client and report less mutable

Keep returning a pointer in OpenReport. Apart from that, do not return
pointers and make sure st...

b603f14d9a04e083e9551b87303793aaf205c814 authored over 4 years ago by Simone Basso <[email protected]>
probeservices: don't duplicate jsonapi fields

Instead, use embedding, so that we don't need to write tests
asserting that we pass downstream t...

bb4a0a6077f6d4af108c42e1bf6a00a8ff7cd09f authored over 4 years ago by Simone Basso <[email protected]>
jsonapi: refactor the implementation of tests

This is part of https://github.com/ooni/probe/issues/887 where I
determined that we needed to do...

360dc51072d1cafe009005ad1188a319b9bf68bb authored over 4 years ago by Simone Basso <[email protected]>
jsonapi: do not log request/response body

Closes https://github.com/ooni/probe-engine/issues/270

Part of the refactoring that I think it'...

e0722d2ae091055673e157d9e71669ae925dd78d authored over 4 years ago by Simone Basso <[email protected]>
jsonapi: don't allow Client to mutate itself

There is no reason in doing so. Mutable state leads to code that is
more difficult to analyse, h...

59b3494d52699f88f5d4e1b473c7a25fb499c15e authored over 4 years ago by Simone Basso <[email protected]>
jsonapi: improve the documentation

Part of https://github.com/ooni/probe/issues/887. I determined that
we needed some refactoring o...

2af83deb915656f49f932a68a33223222e1b0145 authored over 4 years ago by Simone Basso <[email protected]>
jsonapi probeservices: allow cloudfronting and tunneling (#601)

This diff modifies the code in jsonapi and probeservices such that it
is now possible to config...

6892aafff2f4130862e036ee6d979839b3f32071 authored over 4 years ago by Simone Basso <[email protected]>
netx/dialer/proxy.go: allow extemporary proxy (#600)

This diff allows us to use the context in order to setup an extemporary
proxy that is only vali...

4a39a3bce5d91c1a60d6185220cbb55079b51933 authored over 4 years ago by Simone Basso <[email protected]>
refactor: merge bouncer and collector into probeservices (#599)

To implement https://github.com/ooni/probe/issues/887, it helps to have all the
logic for speak...

156e55cbe7daa2ba7209a0aa408ccac67b5be466 authored over 4 years ago by Simone Basso <[email protected]>
Merge pull request #597 from ooni/release/v0.11.0

Release 0.11.0

65e4e617f1d6a018dfa78ae36752635397644aff authored over 4 years ago by Simone Basso <[email protected]>
Use most recent MK build and update sprint releases todo list

We want to make sure we perform this check at every release so that
when we build miniooni we us...

9eb8177964d9fdc82f7b79c0f93db0e597c01a60 authored over 4 years ago by Simone Basso <[email protected]>
ndt7: remove dependency from m-lab/ndt7-client-go

Necessary because the upstream dependency doesn't build on Windows anymore.

See https://github....

beca6853be575bfc9e34a108b8329a68ea483e7f authored over 4 years ago by Simone Basso <[email protected]>
Pin to the latest release of ooni/probe-assets

c7d27fcb650133857b861f91331d72bb00c3fc3e authored over 4 years ago by Simone Basso <[email protected]>
Bump version to 0.11.0

0d05001034727115c457222f127092fd6ac05dc6 authored over 4 years ago by Simone Basso <[email protected]>
Update direct and indirect dependencies

4b295fe308e777602c09754698e678b0ce7301ee authored over 4 years ago by Simone Basso <[email protected]>
psiphon: collect performance metrics from arbitrary URLs (#596)

This diff contains the following changes:

1. rewrite psiphon to use the urlgetter experiment
...

45526057e92a1b5d8e0f1cf879792988f6c80e79 authored over 4 years ago by Simone Basso <[email protected]>
netx: use gocertifi in new code (#591)

Closes https://github.com/ooni/probe-engine/issues/335

cbb7b295c1f13eee30a53fc5c6404524c55e97bc authored over 4 years ago by Simone Basso <[email protected]>
Use random HTTP header for GeoIP lookup (#589)

Closes https://github.com/ooni/probe-engine/issues/578

911040ad7b9b8bc5c92c73c6c9ee8fcafaf80f30 authored over 4 years ago by Simone Basso <[email protected]>
ndt7: use m-lab location services v2 (#588)

While there, implement throttling for GitHub action builds.

Closes https://github.com/ooni/pr...

edd71c0450f1713038752fc351c73422ec766e6e authored over 4 years ago by Simone Basso <[email protected]>
Implement the tor tunnel (#585)

This implementation currently attempts to use an external tor
command and fails if not found. H...

43c7a7b0d073af449ebce3121708669037cc29a8 authored over 4 years ago by Simone Basso <[email protected]>
routine-sprint-releases.md: remember to update host header

443e98e8a6c857176ed67cc09e65e8e192c88e30 authored over 4 years ago by Simone Basso <[email protected]>
fix(netx/archival): include TLS server name (#580)

Part of https://github.com/ooni/probe-engine/issues/547

52df3cf408db2ccf5b397a7bf4a11d7ad8e1ba2f authored over 4 years ago by Simone Basso <[email protected]>
Merge pull request #577 from ooni/issue/547

Make library for miniooni and write basic tests

bee591d49f59be6ae23ecc6a127074bb3697ceff authored over 4 years ago by Simone Basso <[email protected]>
libminiooni: set the correct version number

12e0ab8d1335ead20fcf25c6e55974bc2ca38837 authored over 4 years ago by Simone Basso <[email protected]>
libminiooni: add basic integration test

It just verifies we can run the example experiment without crashing
because some options/tables/...

ea6820415b3509bfce3d4fac5bb97313f9fa6d46 authored over 4 years ago by Simone Basso <[email protected]>
libminiooni: panic on failure and let cmd/miniooni exit

This allows to write some basic tests of libminiooni.

e6dcf40ef7e34b4ed6c4f69a644dd1005a946abe authored over 4 years ago by Simone Basso <[email protected]>
libminiooni: decouple the real main from globals

5443fcbf2175745c5970640fe90e293f18f178e9 authored over 4 years ago by Simone Basso <[email protected]>
libminiooni: disable -g and stop if we cannot get the user IP

Closes https://github.com/ooni/probe-engine/issues/297

4974d6cbb5cb77dba455c0106c78b610a3820b45 authored over 4 years ago by Simone Basso <[email protected]>
libminiooni: more cases where we avoid explicit if statements

f4484582137da0dba69b37989afd4d3b4fbd7e8f authored over 4 years ago by Simone Basso <[email protected]>
libminiooni: allow a not-particularly-long line

7bd69292e471b55b2232e2daaf1715cc2b1e0078 authored over 4 years ago by Simone Basso <[email protected]>
libminiooni: add syntactic sugar to handle errors

This reduces the number of unnecessary code paths to be tested.

d7d928f48122868fc77b479bbdc4db6c2cd467c5 authored over 4 years ago by Simone Basso <[email protected]>
Allow other Go packages to import/use the miniooni CLI

Part of https://github.com/ooni/probe-engine/issues/547.

13fbe9e9dcdb72d5485cea9ab22c09fc7b3bfe8e authored over 4 years ago by Simone Basso <[email protected]>
experiment/urlgetter: include DNSCache in TestKeys (#575)

This simplifies analysing the measurement JSON.

Part of https://github.com/ooni/probe-engine/...

410e4fb2b9777a09a111695bc2a07913aae4928a authored over 4 years ago by Simone Basso <[email protected]>
netx: introduce resolver that handles IP addresses (#574)

We want to map an IP address to itself rather than failing.

Part of https://github.com/ooni/p...

b387064ee5f297fdfd7e1c9b5f83cc32490096d3 authored over 4 years ago by Simone Basso <[email protected]>
urlgetter/dnscache: allow multiple entries per domain (#573)

Part of https://github.com/ooni/probe-engine/issues/359

0b964f57d5f3060720dc052f9e3fd0aa26f4125e authored over 4 years ago by Simone Basso <[email protected]>
netx: make sure we log addresses that trigger bogon filter (#571)

Part of https://github.com/ooni/probe-engine/issues/547

13b6db5059c01bd779901755d0eaad31d7b5432f authored over 4 years ago by Simone Basso <[email protected]>
netx, experiment: implement prefilled DNS cache (#570)

Seems to be required when running HTTP experiments where we have
previously already resolved th...

6c146118a383f3d23d10dc72f0f6dde60863495a authored over 4 years ago by Simone Basso <[email protected]>
netx, experiment/urlgetter: allow to disable TLS verify (#569)

Part of https://github.com/ooni/probe-engine/issues/543

933a5d67e772a7539ecb84c9e2bcb8fb7bd90d7a authored over 4 years ago by Simone Basso <[email protected]>
netx: reduce toplevel HTTP errors to OONI failures (#568)

* netx: reduce toplevel HTTP errors

This is related to https://github.com/ooni/probe-engine/i...

b6dbb2d83c70d81ce21488f96f34eff18348b45d authored over 4 years ago by Simone Basso <[email protected]>
experiment: add the new urlgetter experiment (#567)

This is the basic building block on which to rewrite most other OONI
experiments. It handles di...

8ecd7eed0d7ce4c572f9ffe8fd5abd31a8203163 authored over 4 years ago by Simone Basso <[email protected]>
netx/httptransport: make construction more flexible (#566)

While there, write unittests.

Part of https://github.com/ooni/probe-engine/issues/543

9cbfb123ebeebef5df0e9c93712b067d36d7d09e authored over 4 years ago by Simone Basso <[email protected]>
miniooni: also accept boolean options (#565)

Part of https://github.com/ooni/probe-engine/issues/543

2d500b37f284d291ed57b8db54baf88570653e40 authored over 4 years ago by Simone Basso <[email protected]>
netx/internal/errwrapper: scrub IP addresses from errors (#564)

To this end, vendor code from keroserene/snoflake with attribution.

Closes https://github.com...

950749ff10e1ac92f046cf5e61cd2bd3ba5b0b9e authored over 4 years ago by Simone Basso <[email protected]>
netx/archival: generate the OONI data format (#561)

* netx/archival: generate the OONI data format

Part of https://github.com/ooni/probe-engine/i...

c3cf36ce686ba3f678fc1b54335bb022201b4c4f authored over 4 years ago by Simone Basso <[email protected]>
netx: restructure http events for easy saving in OONI format (#560)

* netx: restructure http events for easy saving in OONI format

Part of https://github.com/oon...

6ba229608463ee9ff4d763a7a138975356582d64 authored over 4 years ago by Simone Basso <[email protected]>
Merge pull request #559 from ooni/issue/543

More netx/httptransport refactoring

049d6b212b407c28f096182cdee3d24b8256987b authored over 4 years ago by Simone Basso <[email protected]>
netx/httptransport: save body snapshot for OONI measurements

This is part of https://github.com/ooni/probe-engine/issues/543

0f7c97a8a0808b818a1c794f4ffbac58e1ee28d5 authored over 4 years ago by Simone Basso <[email protected]>
netx/httptransport: rename the default saver transport

Part of https://github.com/ooni/probe-engine/issues/543

ac608392371e2f8cf10a6bc57223c94a988d56f5 authored over 4 years ago by Simone Basso <[email protected]>
netx/httptransport: bugfix: save r/w events

Part of https://github.com/ooni/probe-engine/issues/543

2a4e20b0c7c506f37a3ef723810c49fb3f77b993 authored over 4 years ago by Simone Basso <[email protected]>
netx/httptransport: allow to configure different savers (#558)

Specifically for DASH, we're only interested in saving dials.

Part of https://github.com/ooni...

dff2cef890b16f7561cfbc5aa7cc33fb6891c788 authored over 4 years ago by Simone Basso <[email protected]>
netx/httptransport: reintroduce http/httptrace (#557)

In https://github.com/ooni/probe-engine/pull/556 I was optimistic. It seems
it's difficult to g...

6332abd3eb5ed66904cc58b9d6cdbd1fe4860b33 authored over 4 years ago by Simone Basso <[email protected]>
netx/httptransport: stop using http/httptrace (#556)

Using http/httptrace leads to tricky code because of the context and the
fact we may mistakenly...

d30a621cce307ea8177e471e29a1e778e869d0e7 authored over 4 years ago by Simone Basso <[email protected]>
netx: decouple TLS from saving R/W events (#555)

We may want to perform TLS measurements without saving R/W events and we may
want to save R/W e...

cd30b33f221acc6d776928c9781ac90ce261dc0e authored over 4 years ago by Simone Basso <[email protected]>
Merge pull request #554 from ooni/issue/550

ndt7 and dash: save mlab site, include user-agent in ws handshake

78677d0b6437facc6801b3cfd8cb0a6ef763b0d3 authored over 4 years ago by Simone Basso <[email protected]>
ndt7: include User-Agent during handshake

The spec says we SHOULD do that. Noted while working on previous diff.

781bd9a9f8d60b42ddaccbbb516f675ecc4b94fe authored over 4 years ago by Simone Basso <[email protected]>
dash, ndt7: save mlab site we're using

Closes https://github.com/ooni/probe-engine/issues/550

Closes https://github.com/ooni/probe-eng...

204e82e38140559be96b430ff443d00295e4d956 authored over 4 years ago by Simone Basso <[email protected]>
Merge pull request #553 from ooni/mobile/fixes

oonimkall: address issues reported by @lorenzoPrimi

9faa5f3c6ba2f98e39b9ba1841818af3ec4992bf authored over 4 years ago by Simone Basso <[email protected]>
oonimkall: tell runner to always deliver events

Not delivering events will confuse the mobile app. Rather than timing out,
which quite likely is...

03ab6b7e7aee33aa6bdc0561087b1b909c4cfc93 authored over 4 years ago by Simone Basso <[email protected]>
oonimkall: honour privacy settings

Closes https://github.com/ooni/probe/issues/1129

675897d4ca7844ecd08d65bb3bcbcd0f7c6d1ee1 authored over 4 years ago by Simone Basso <[email protected]>
Merge pull request #544 from ooni/release/0.10.0

Release v0.10.0

Part of https://github.com/ooni/probe-engine/issues/504

4bcf3efabfa0589ed0a63a9fdcfba02a188e5657 authored over 4 years ago by Simone Basso <[email protected]>
Update direct and indirect dependencies

Part of https://github.com/ooni/probe-engine/issues/504

1ee00f5b62878e0bb622d512ecad3490caf16f7d authored over 4 years ago by Simone Basso <[email protected]>
version/version.go: bump version number

Part of https://github.com/ooni/probe-engine/issues/504

2b6d4e2ec7272049d27f073a9191f512457058ef authored over 4 years ago by Simone Basso <[email protected]>
httptransport: don't save events twice (#542)

Bugfix developed as part of https://github.com/ooni/probe-engine/issues/509

While there, impl...

3fd5a189757b3d6e99156b7797da1dbe78b85f6d authored over 4 years ago by Simone Basso <[email protected]>
resolver: collect transport's Network and Address (#541)

These are useful bits of information when filling a measurement
because they allow us to unders...

2a79e7f824821b6b1d2d6109d2537881d6e13b6c authored over 4 years ago by Simone Basso <[email protected]>
Introduce a caching DNS resolver (#540)

There is no cache aging. If a resolution is successful, then we
do cache the result. This is me...

70278838b88775485ad9b2a5670bf0373a1bc591 authored over 4 years ago by Simone Basso <[email protected]>
Measurement += ProbeNetworkName non standard field (#539)

This is non standard. We don't need to add it to the spec. It is just
useful when you are vetti...

7c5a269e3ddafda8892a2d61396dc8fba1053341 authored over 4 years ago by Simone Basso <[email protected]>
ndt7 and dash can run over psiphon (#532)

The design is that we have a tunnel managed by the session. We may ask the
session to possibly ...

a91bf6e45c649811478c1bd61c355ba0951a87cb authored over 4 years ago by Simone Basso <[email protected]>
Rework code that manages proxying (#531)

The rationale of this change is the following:

1. honouring `HTTP_PROX`Y is increasingly weir...

5236d4bd734fb8cde80eaf04e16df6dcc8ae01fd authored over 4 years ago by Simone Basso <[email protected]>
Make the output of DASH and NDT more pleasant (#530)

Part of https://github.com/ooni/probe-engine/issues/501

22f37543a5781042224a0b56347f60f0c5b1e0bf authored over 4 years ago by Simone Basso <[email protected]>
dash: bump the version number (#528)

The implementation has changed significantly.

Part of https://github.com/ooni/probe-engine/is...

f295e4f94f11ba75580e8c5806ff413e583ac73d authored over 4 years ago by Simone Basso <[email protected]>
dash: blend together existing code (#527)

We don't have anymore a boundary between the external neubot/dash
client and probe-engine code,...

4074dc56cb2b92e0d961858143e1112e1098deee authored over 4 years ago by Simone Basso <[email protected]>
dash: don't report error after download started (#526)

This is similar to ndt7. Treat this kind of errors more
liberally and do not fail the whole exp...

4ec09dfe5474fb357bb2b9bb60e63c46ab840477 authored over 4 years ago by Simone Basso <[email protected]>
dash: simplify construction of http.Client (#525)

Part of https://github.com/ooni/probe-engine/issues/501

3f52e7865e9787c4da0c45555939e5d909d1316b authored over 4 years ago by Simone Basso <[email protected]>
dash: remove unused/unimportant code (#524)

As part of merging the existing codebase with the neubot/dash client, let's
also dispose of the...

36f9d1578b160e2810e9dcaf592c48a38cff5560 authored over 4 years ago by Simone Basso <[email protected]>
dash: save and populate the connect latency (#523)

Part of https://github.com/ooni/probe-engine/issues/501

ea566eb48996b831dc9550f0e739db9087a3f26b authored over 4 years ago by Simone Basso <[email protected]>
netx: implement events saving w/ decorator pattern (#522)

This seems to be a better pattern for saving events that involves
doing much less magic with th...

633d2a1737cf90c2ce9e823a92b748c329852ad6 authored over 4 years ago by Simone Basso <[email protected]>
dash: report speed in kbit/s (#520)

Part of https://github.com/ooni/probe-engine/issues/501

7c252c78e3e75be72fb2e8ab5a3136ab5c8ae4cf authored over 4 years ago by Simone Basso <[email protected]>
dash: make sure we correctly count bytes (#517)

Part of https://github.com/ooni/probe-engine/issues/501

It should be noted that this solution...

6b348e57b84b2d3a72a5f251764d1a1b1e2fd186 authored over 4 years ago by Simone Basso <[email protected]>
dash: print server name after mlabns call (#516)

Part of https://github.com/ooni/probe-engine/issues/501

95d4ec020a8f65463c848e461b65ddd9b2a8ed69 authored over 4 years ago by Simone Basso <[email protected]>
dash: use our internal mlabns client (#514)

Part of https://github.com/ooni/probe-engine/issues/501

2d410bdd242f5d12d9acbe73808af448dfcccabe authored over 4 years ago by Simone Basso <[email protected]>