github.com/ooni/backend commits | Ecosyste.ms: OpenCollective

Do not parse vanilla_tor test_keys if test_name does not match

e7b7ede8af96ebf418dfb5aa6dd51d96b18bd10a authored about 6 years ago

DNS TTL=0 is "valid" TTL value. At least it's not exceptional

c7acf01562b0c8bf4f82896685d3625040e91405 authored about 6 years ago

Use simhash cache on data reprocessing

It gives ~10x speedup of reprocessing on my machine

c8e6d3a4a5ca7b3f980da51bb659d93b9928cda9 authored about 6 years ago

Merge pull request #126 from badblob

Ingest `badblob` to track failures

fa7de28cd1baf30c84bca5980268561b4f5ba015 authored about 6 years ago

Ingest `badblob` to track failures

Non-ingested `badblob` blocked ingestion of data that is referenced from
Uganda report[U16] and,...

62f30b74e4b080385a452bf843530aa0ddc751e8 authored over 6 years ago

Fix typo

b1e17fbae563f1da6f961ac38331ed79c5366221 authored over 6 years ago

Merge pull request #124 from simhash

centrifugation: calculate simhash(body) and simhash(text(body))

7a549ee83dac396a107f15f3762f42005f84f6f2 authored over 6 years ago

Drop stale build script

0fec08f655766001f574ebab02bad54a921fe9c2 authored over 6 years ago

centrifugation: calculate simhash(body) and simhash(text(body))

It'll enable heuristics that rely on page similarity to mine more
blockpages our of existing OON...

8e14b20ec368572c0bb831fb958bcc70eb9108a6 authored over 6 years ago

Move some bits of code to `oonipi` module

Also, re-run mubench/hash64_shingles to check for possible regressions.

d8dde72d2622fc25488d82e27d5ca560822bc2d9 authored over 6 years ago

shovel: add rsync & ssh

11a5d747f796ee518841a23e4bfc121290c682e1 authored over 6 years ago

Fix API docs links

210e8603aea57e03e3fc6066d5967c2bf53879a9 authored over 6 years ago

Add loader for stats

e83b09ec3e8430b835dce8e2f55637ba7a770022 authored over 6 years ago

Update the charts on the stats page

* Only show the last 24 months
* Don't show the unfinalized month
* Round the last day of the mo...

6584d5b354ee6e3f1d04dc6b96d9c77c22dd19bc authored over 6 years ago

Merge pull request #56 from ooni/stats-speedup

Fix https://api.ooni.io/stats speed

6c0dea143b94fb1f5ed26441b212977c2c8ab3d3 authored over 6 years ago

Merge pull request #120 refactor/ooni-uuid.md

f0f05668d3544143a079e79884fb361d7d0b4ba3 authored over 6 years ago

Add note on the fact that the offset may change

eb2faa7dfa0621c11120a0bb2cc62171c294727f authored over 6 years ago

Move "keyed hash" estimates to separate section

61ca75d248a279e7ac4f1ff3defb84bf445e5d18 authored over 6 years ago

Make a sentence more clear

e64ded763f6ab4966465d8e105f3750e9bc82ef4 authored over 6 years ago

Reorganise some of the text in ooni-uuid

Make some minor edits and formatting to the backfilling section

1545ccef92c8f9962e66b7fa026b5f10bfbc61da authored over 6 years ago

Bump version in other place too

b6ab708b78dcd46e731ed0d549299cecf658526f authored over 6 years ago

Add changelog entry and bump version

a704b90892cd0da4944d0c335c1cb18372d24a5d authored over 6 years ago

Fix deprecation warning

09114a56e1c6576714c575528ef6080f6187dfbd authored over 6 years ago

Add validation for the test_name

37ac5d3521285ec5dcfbe010a681cf29558ac3f2 authored over 6 years ago

Fix bug in filtering by failure=true | failure=false

b574b4cc7dde56e343670ab41cb974a8506946bf authored over 6 years ago

Add the reason to have "namespaces" -- to distinguish backfilled measurements from stamped during _transition_

6986358f2ee607c9243cf8b7fc806bfe545235be authored over 6 years ago

Fix typos

ac56b4e3c3b9288eccaa488720f376383caa3823 authored over 6 years ago

OOID: on 24bit and 20bit counters

bc2872b5f555db5085508b4f5140d1fc0d8ad415 authored over 6 years ago

ooid: estimate max-report-size in bits

500d7aed91ffb1dd011fc11e4353ae100c6030c9 authored over 6 years ago

ooid: fix typos

91a113b6f63faae4a062e26c7a7543fb3ec31f53 authored over 6 years ago

ooid: add total number of measureents

c82fda2ad3f7024f0e062ba05418a07928e6ef37 authored over 6 years ago

doc: on OOID, OONI measurement UUID

5a57a1e07afd5d73c30f772171eee4c173df9f41 authored over 6 years ago

Fix /data .gitignore

e2e09d709df2ac78d4df3f400b12a8ec2b92a7d2 authored over 6 years ago

cleanup_uploaded: to drop uploaded `canned` and `reports-tgz` files

a083c7d878eea1728db8685edd94850c122e81c5 authored over 6 years ago

cleanup_reports_raw: update with new dependencies (local reports-tgz & canned)

0b6cb2e7e4c0360374ecf21eff954e06226640f7 authored over 6 years ago

Various fixes to the Dockerfile

a57ac59a3ff5e361c02087bb832e500988e41d85 authored almost 7 years ago

Add basic Dockerfile

0b0c8b3d0b81cd4df6c9bbc76d339ea7e6f811c9 authored almost 7 years ago

Fix https://api.ooni.io/stats speed

532c35cdbb2639476b5011ffca1e97382a291644 authored almost 7 years ago

shovel: convert tar `--add-file` to `--files-from` because of lots of tiny NDT measurements

cd75d38165dcc9e08d371f2011590a6870238620 authored almost 7 years ago

s/measurement/api/ in Readme

28b0143efcbe5adac72cfe430ade590b6acbbfce authored almost 7 years ago

Merge pull request #51 from TheTorProject/fix/disallow-order-bys

Fix/disallow order bys

7360a5475ca0f63e544db6fe963706b6813b9ee7 authored almost 7 years ago

Add comment minLength=3 for `input`

1052282fc955796980a6a3f9c1cde46bd756f69e authored almost 7 years ago

The minimum length of the input can be specified in the swagger spec

13b5210362b543c0c6243931573fc5a059e9f517 authored almost 7 years ago

Use INNER JOIN if `input` filter is enabled for `list_measurments()`

8410e32cbfad1065917c47014fdfa682942481a0 authored almost 7 years ago

Update jquery dependency

462c6c27a3030b88b27ba71324453703dca2cd4d authored almost 7 years ago

Fix documentation bug that claims we support `order_by` values that we don't

3aefea7259963aacb0ac1dd2a6d25cde1e187615 authored almost 7 years ago

Merge pull request #50 from TheTorProject/fix/order_by

Order measurements by test_start_time by default

d5ef842c568e59966d29bfd8cb3a25dad08d88a3 authored almost 7 years ago

Order measurements by test_start_time by default

This aligns the implementation to the spec

7d70c74eeeac997814d15dda55138790c8b8c34a authored almost 7 years ago

Add `tar_reports_raw.py` script to create reports-tgz bucket

6c3f7cefbb2713a816b32c29536c35a73c40b709 authored about 7 years ago

Merge pull request #48

Use Request-Id header, forward it via HTTP and to postgres

a503c1185a2306dbd3ecd8ecf621a74078dff960 authored about 7 years ago

Use more generic `X-Request-ID` instead of `Request-Id`

488187497bedeff2bc2a9d0b85ca9bce85a3e625 authored about 7 years ago

Add DATABASE_STATEMENT_TIMEOUT setting, track client PIDs with application_name

PIDs of processes are tracked to understand strange disconnect /
re-connect pattern of `measurem...

bb97c9b77fa015c74c0fabe20ee6036a4b5ecb46 authored about 7 years ago

Parse Request-Id, forward it via HTTP and to postgres

9610c67d62342f1f6b43d59349fe8b83ead6d4ac authored about 7 years ago

Hotfix: moar test_name enum values

4d43001a325500bda03dfc72645668352193f176 authored about 7 years ago

Hotfix: moar test_name enum values

179cdb2473b8b4b39327ebc622e61af40cc70592 authored about 7 years ago

Merge pull request #98 from feature/tor-metrics

a950764394637c2066e31d15839837cfd447c303 authored about 7 years ago

Make close() one-way ticket

26dc901ea497349fabc624d76931884311d93ee6 authored about 7 years ago

centriguration: fix badrow handling

ca143c08d5b6cf61d9c9b384c1d5942f1cb14f90 authored about 7 years ago

Add missing file to Dockerfile

a924ddfacc34b90ca24d8282fb9ebe2a08ccc734 authored about 7 years ago

More good comments

90445631e86fac3c1dd053eb616a576c64e0a164 authored about 7 years ago

Merge pull request #47 from TheTorProject/release/1.0.3

e15c065b84b45cd0401cf990ebfad4c9c7e8eccf authored about 7 years ago

Bump version, add changelog entry

0a82946850a90326140e25b7f8e467fcc35dbdb6 authored about 7 years ago

Fix CENTRIFUGATION_BASE_URL

2b0746829958f4a53c51caf30ad798c917495bbb authored about 7 years ago

Add changelog entry for 1.0.2

6fa7c67020c3764056c0b8d3fa34464679cbdda3 authored about 7 years ago

Fix typo in env var getting

3ae4bfd763085ce0a0e9450c43e713420e23686b authored about 7 years ago

Merge pull request #46 from TheTorProject/feat/unbreak-explorer

0c6b10882ec121bf1c2ae9e0cab67e60286b7f08 authored about 7 years ago

ooni-measurements (also known as ooni-api) has 4 (four, quattro, τέσσερα, четыре) known environments

798fd19620fd6e0f49b2aa113babaa80e959d41f authored about 7 years ago

Bump to 1.0.2

eaa13e3e8686719d1c9f0275d50913d3ba180c5b authored about 7 years ago

Some more safeguards while fetching autoclaved files

64081d1c911954fe9a2f5144763cb020141041dc authored about 7 years ago

Some devenv pragma

(cherry picked from commit f16a995e4421a9b99615c612e34374e7ee888500)
Signed-off-by: Leonid Evdok...

9662a12fb3b19764cea0c88215ad0afbccc8b96f authored about 7 years ago

Some safeguards

(cherry picked from commit ba1d5d48d4b9d6dbde264e7d625203d63a9c5eb2)
Signed-off-by: Leonid Evdok...

ba03b2897bc48692808df0898c7b3c03be15c073 authored about 7 years ago

Merge branch 'fix/download-urls' into feat/unbreak-explorer

db9a65e4b7be9530f16f157a37da274f4932599f authored about 7 years ago

Fail fast when python is < 3

41b08c3da63b03440fd3ac2d8def5ac963bc021c authored about 7 years ago

Drop dependency from six

d81dad500728d4956fa0950b455a2dc59e74c3ff authored about 7 years ago

centrifugation: split steps `ingest`, `reingest` and `reprocess`

a86e4fe42dec869b02618ba2c3e282812052cab9 authored about 7 years ago

Add more good comments

d31a202c79d8adc7f36e562a476d8bbc011a1c0c authored about 7 years ago

Add fetching of the resource just needed for the data viz of the country

a33def36defd05e617a2c79675e2a5c6bcadd661 authored about 7 years ago

Add endpoint for detected blockpages

e297b9f066dda76ea46b06df3cf421d8c7b89df9 authored about 7 years ago

Add endpoints for counts

991dbf222a1a18535bffdac59cfbc12151839e60 authored about 7 years ago

Typo

3aafc4a524084e9717f0fad845bb2fa6f31b4b2d authored about 7 years ago

Fix a typo and improve if clause based on @darkk feedback

53fd39deb466743f1f92f4f11d56e5fd05150870 authored about 7 years ago

Add private API endpoint for blockpages

ee14034cbbfe3cd29cbc00a3962a0d73df8830a6 authored about 7 years ago

Add backward compatibility layer for file paths without a bucket

258abc2f1983e8f4bb4c5c28849f0b2727bca47f authored about 7 years ago

Increase the threshold for query time logging

1c1df508f6e94241dde2c92b7052576a248f499b authored about 7 years ago

Attempt to extract the traceback in some cases

a01bc6ccaec329d523378ef5b61f72960e15115a authored about 7 years ago

Fix typo

210292ec8ff99ed1c7b0ffe0d9b9bfbb95c1a923 authored about 7 years ago

Parse `tor_log` to ease further data mining

efb274c0d6119acdb3ae1d588279684f1ab59437 authored about 7 years ago

Add some styled 400 and 404 pages

62975402613f4f1a933a310f33221c85ed758ac7 authored about 7 years ago

Add better handling of 500 type errors

f3fb9fd75aae557f7e0ad4b5521a4020a39a2fcb authored about 7 years ago

Merge pull request #41 from TheTorProject/hotfix/1.0.1

(it was deployed on ~ 01 Oct)

3003e8ebdcac0d11fe459660405ab4ba051bb3c6 authored about 7 years ago

Add vanilla-tor sql scripts

ac8e349b86246c6c50c4464bd971e5e43b4c277c authored about 7 years ago

Integrate feedback from @darkk in https://github.com/TheTorProject/ooni-pipeline/pull/97

5e8a091cdb65e0dc2aeb5265002c86a28672851e authored about 7 years ago

Add more docs on how to write custom feeders

b8125a830a3bdd29993a8faf658d89616fe7574c authored about 7 years ago

Fixup number of columns

727d6e1900a8ddeb9c9153790e269c0a887765ac authored about 7 years ago

Add Feeder for vanilla_tor tests

902e6751340dd515096214f74c739751c9ddca55 authored about 7 years ago

Add more information about schema changes in the Readme

a133595933a9e636f7bd1e7f7cecb85d689ac4db authored about 7 years ago

error is actually text

105d17c6045799fe910276a240a8dde2598f304a authored about 7 years ago

Check for the vanilla_tor test_name

b6c90e85cb1c27f02fd88c5480a224385b1a488e authored about 7 years ago

Update README regarding `backfill` surprises

798796775849655f0230ba4c51f6f8d2f3278b9e authored over 7 years ago

shovel:0.0.19 - S3 upload resumption is fixed, fixing uploading to an empty bucket

84e738e71ac65cdc62cb816c91dc0e6d776b6448 authored over 7 years ago