Ecosyste.ms: OpenCollective
An open API service for software projects hosted on Open Collective.
github.com/ooni/2022-04-websteps-illustrated
websteps: winter 2022 edition
https://github.com/ooni/2022-04-websteps-illustrated
This matters when we're rerunning previous measurements using the cache.
Without this fix, the ...
fb5cf82ce1a3f820bca472e65716b560272f09c0 authored over 2 years ago by Simone Basso <[email protected]>
These changes are based on my trying to make sense of the
several measurements I'm using as test...
1. we need to rewrite the IDs when we pick results up from the cache
because the ID in general a...
This is required to re-run measurements using the cache.
While there add support for asbolute t...
0e4601110a424b5ac958d43c0b3e165d124dc782 authored over 2 years ago by Simone Basso <[email protected]>I should have added this code to miniooni long time ago.
We're using the same format of minioon...
e04b5a8b553a4b553b521eed47126f879c7fc735 authored over 2 years ago by Simone Basso <[email protected]>7aeba5751a101005c669895a6027d659785ee473 authored over 2 years ago by Simone Basso <[email protected]>
This seems one of the missing bits to ensure that, when we're
repeating a measurement, we're not...
We _also_ need to cache dnsping results to make measurements fully
reproducible later in time. W...
This diff introduces changes allowing one to use the websteps command
to fill a predictable, com...
This command line design is more flexible.
556a91c9a5dd50b57f777febf4df6b6d0ff6a958 authored over 2 years ago by Simone Basso <[email protected]>
If we only use one, we're not able to compare the DNS resolver and
the system resolver to say wh...
cbe771fcc811e79dd59fc478de0094ac3cf2a2a4 authored over 2 years ago by Simone Basso <[email protected]>
This should help me implement my plan of having two local caches
for re-running measurements loc...
9c8f60c6873c18afd9bab7586bb81e8a132ac5d8 authored over 2 years ago by Simone Basso <[email protected]>
5855bdf23312c824ae28b369978f92aa02861db5 authored over 2 years ago by Simone Basso <[email protected]>
1. stop relying on hashing because we're not convinced
2. cleanly split each part of the analys...
604ee8197e8c14f54c5bb4121f5c1f3da6fa6fb8 authored over 2 years ago by Simone Basso <[email protected]>
This path was a bit too tricky, so I'll try a simpler path that does not
require me to separate ...
Inspect is informing the user we're doing an activity.
Scrutinize is telling the user we're loo...
4a18ab80daedc223841f4407a6f31b020bcacf10 authored over 2 years ago by Simone Basso <[email protected]>Otherwise, we may end up missing some messages.
849058d7417153808bd386ef5db2dc674a3b65ba authored over 2 years ago by Simone Basso <[email protected]>1ebfb1c981a42384f3d3a772bbe04320d88143c1 authored over 2 years ago by Simone Basso <[email protected]>
3f631ad6aa4b8666d0c3b2959aef49b2eb93aedb authored over 2 years ago by Simone Basso <[email protected]>
It's very seldom used and so let's just not use it.
21e57fe5a5052db1d775321c829418d5ea873e19 authored over 2 years ago by Simone Basso <[email protected]>23b45a348245d62e57f2a75b3640ede91011b98a authored over 2 years ago by Simone Basso <[email protected]>
This message is an info message associated with 🧐.
487709611a72c3f0a33b9b0e3d1289712b36a8e1 authored over 2 years ago by Simone Basso <[email protected]>
The previous code was not obvious and I could not fully explain why
it was working or non-workin...
In greedy mode we stop following redirects as soon as we find
some signs of internet censorship....
If we also use dnsping, then analysis is going to have a hard
time because some TH endpoint meas...
1. ensure that the probe is going to test in its additional round
only new IP addresses returned...
After reasoning a bit more on it, it turns out that
d46101418537113d15c05d7309f4de8fcae28242 is ...
We can reduce the complexity of Int64 by just ensuring its int64 value is
the first element: thi...
With embedding, every time we add a new method, we don't get a compiler
error, which means the n...
d46101418537113d15c05d7309f4de8fcae28242 authored over 2 years ago by Simone Basso <[email protected]>
We encode emojis inside the loglevel and each consumer chooses
whether it wants messages with em...
We now have a ring buffer where we store recent log messages, which
allows the mobile app to see...
There is still _more_ work to do in this space, like determining the
full list of acceptable red...
4572b55f445860a19bffb4c90d9e2cbb5456c33b authored almost 3 years ago by Simone Basso <[email protected]>
9dbcba1cb3d9236c28f43fec44d236b6680911b2 authored almost 3 years ago by Simone Basso <[email protected]>
This commit replaces model.Logger instances pretty much everywhere with
calls to the logcat pack...
86d8446aea75cb4c8905e9015db1475bd4228af8 authored almost 3 years ago by Simone Basso <[email protected]>
This query collects extra information useful to further
disambiguate possible DNSDiff cases.
9e8c43256cb79d7dce8c6e7c2a6ee72426edda3d authored almost 3 years ago by Simone Basso <[email protected]>
If we cut the name we cannot see what was matching. I am also
wondering whether we should invest...
1. we need to serialize headers as a list rather than as a map
because in golang maps have unpre...
The correct referer is the previous URL rather than the location.
We should not pass extra head...
4439c047ff0917409c758790947154b7cbb97a36 authored almost 3 years ago by Simone Basso <[email protected]>41c38a57f2b25f0538257b9c09679605b8387286 authored almost 3 years ago by Simone Basso <[email protected]>
af8a947eaaa68698ffa2c1f03c29c011b93114c6 authored almost 3 years ago by Simone Basso <[email protected]>
611b3eedd88870446c1eaaa16e58c230e896f641 authored almost 3 years ago by Simone Basso <[email protected]>
With these information hopefully we're able to say enough on any
given lookup w/o more complicat...
1. collapse probe_initial into the single-step measurement, which
would be annoying to do when g...
b77a9548f74ffcd73c64a723a9703e5334c79433 authored almost 3 years ago by Simone Basso <[email protected]>
a30a1c7152156c33b609aa6785cc55c1ab5f083c authored almost 3 years ago by Simone Basso <[email protected]>
64d95f7a58332f458fe4e9adec96d0fec8dfb457 authored almost 3 years ago by Simone Basso <[email protected]>
7b0c316eaac5e8edcccd5fdba8184314203fd6f8 authored almost 3 years ago by Simone Basso <[email protected]>
2d6eb5cc9c0dab6b0698e823d05c5ca1fd658dd1 authored almost 3 years ago by Simone Basso <[email protected]>
We should not exaggerate with the number of IP addresses tested by
the test helper. Let's add on...
For websites with _many_ IP addresses like YouTube, this would
mean testing a very large number ...
This functionality is very useful when reading logs.
211e6f9b9fb963a2da370d098bae76a19292d05b authored almost 3 years ago by Simone Basso <[email protected]>
I think I had a bunch of false positives caused by the fact that
the test helper was not actuall...
1. cache algorithms were wrong in that they did not prune
duplicate entries but just added new e...
cc11bfd636d562361851584f009e9213a4339b8c authored almost 3 years ago by Simone Basso <[email protected]>
By knowing it, we can emit more useful log messages.
a23341333644092ffd314e8a0fbe85747c1070f5 authored almost 3 years ago by Simone Basso <[email protected]>
We cannot exclude the possibility that a dns measurement and an
endpoint measurement could have ...
a7aa8ee2c4840758efd3b9a197bb3c18fdbc01e0 authored almost 3 years ago by Simone Basso <[email protected]>
Create a summary of an endpoint that also includes all the
relevant options. So, we're now also ...
While caching by IP address is nice in principle, we are going to
possibly have several records ...
This change requires adding support for extracting the CNAME from
a DNS round trip result. We st...
It's easier to reason on a JSON when fields with their default
value are actually missing from t...
We need to know their values to construct a more precise
representation of endpoints where we'll...
We were missing this field, which was not so good in terms of
heuristics for detecting HTTP diff...
751d8eb8a09c2345386bc9a6ab9f9eec7f3eaf6a authored almost 3 years ago by Simone Basso <[email protected]>
71e82c634385dafd2d3480f859da3d83d1a7056a authored almost 3 years ago by Simone Basso <[email protected]>
137c62a8af5a13c7959da5ebe027fc2d783e09c4 authored almost 3 years ago by Simone Basso <[email protected]>
bd2a44352687ddf3f4a7f5ee0042ff5ac243a401 authored almost 3 years ago by Simone Basso <[email protected]>
With a mandatory cache it's too difficult to develop in the
common case where I need to test wit...
Store all endpoints by IP address and check for equality using
the current summary (we still nee...
2b3a8aa556325ccdb27c1bf811e0008914c30c26 authored almost 3 years ago by Simone Basso <[email protected]>
We're now caching on a domain basis. We have improved the rules
to check whether a plan and a me...
When searching for a compatible DNS lookup measurement, we should
first try to find a "same as" ...
It's actually fine to cross compare getaddrinfo and HTTPSSvc as
long as both have gathered some ...
5880741cea96eccb03237cd651614e93c01bab07 authored almost 3 years ago by Simone Basso <[email protected]>
ad17d7a8f6987f38f458c91e1d1c732dc24fd515 authored almost 3 years ago by Simone Basso <[email protected]>
We need to refactor the code until there is a single plan for
each measurement. This is the best...
This diff changes the way in which we create DNSLookupPlans to
produce a single plan per resolve...
These flags allow one to easily specify extra queries to
perform (e.g., HTTPSSvc or NS) as part ...
The idea here is to never expire the cache. This allows us to
test the cache performance. It doe...
This should allow us to retry measurements at a later time w/o
having to really do network I/O.
...
01cc3cea33373f2c7711561e2fa9b8008e152f9c authored almost 3 years ago by Simone Basso <[email protected]>
efb5d220867ab4256e7b9a08716f4b6f2319833d authored almost 3 years ago by Simone Basso <[email protected]>
e7d335ac1d3143de5f7dd72ecce096a24dc97ac1 authored almost 3 years ago by Simone Basso <[email protected]>
The previous diff was not correct, sadly. So we need a new diff.
3ccf1fd9db3758e3d2df6546e9463fa06debe53c authored almost 3 years ago by Simone Basso <[email protected]>
Otherwise the cache ends up being owned by root and it does not
work because the unprivileged pr...
We import the codebase used by Go to maintain its package cache, which
has been modified to remo...
The AbstractMeasurer is an interface that replaces a concrete
Measurer with an abstract version ...
Basically, only transmit fields that are different from their
default value. This change makes t...
Too often they do not work or are censored just like normal
queries, so it seems more economic t...
This diff modifies the emitted data format so that:
1. we only emit DNS round trips
2. we emit...
cdd80f4ffa74552e51bcb6a514841b447ec9d9eb authored almost 3 years ago by Simone Basso <[email protected]>
The advantage of using a (obviously fake) transport for the system
resolver is that we can colle...
a4d669f0060a4fbba4c10d2dcb069e15a6aadabf authored almost 3 years ago by Simone Basso <[email protected]>
At the measurex level, you can ask for a NS lookup by using
a flag that extends the performed qu...
If we're measuring http://torproject.org with the default config we'll
end up using just two of ...