github.com/ooni/backend commits | Ecosyste.ms: OpenCollective

Merge pull request #222 from ooni/ooexpl_wc_input_counts

Add materialized table to improve OONI Explorer performance

4867a2060333dc28c5333380659eb7ab5bc0df5f authored over 5 years ago

Add checks for the ooexpl_wc_input_counts query

33e93ee7db7a81de795621adb7d25c4e33f0e871 authored over 5 years ago

Explicit the column names in the INSERT query comment

1cf2cb08c6c1ca682a57ea85f58459211631ea79 authored over 5 years ago

Rename constraint to avoid clash

4f690db33cc85eb8feb3c4d71832c5d9cb2dc211 authored over 5 years ago

Add index drop to rollback

603a2cbdf247357dfa6c41f0e67f6f18a8b2860b authored over 5 years ago

Fastpath: initial commit ooni/spec/issues/145

742885fd7f2dc3293168ad4b2b0198451ca2ab4b authored over 5 years ago

Merge pull request #221 from ooni/unit

Enable basic unit tests and coverage on Travis

5544e5e4abd3869a87666d099de2449ca48faad8 authored over 5 years ago

Fix the centrifugation update query

25ce207ee99d74bd6595cbe2c6447583c4734782 authored over 5 years ago

Add failure_count column

94868494b722b19b3ef417cc2e8a4d0d8060b81e authored over 5 years ago

Fix missing column

01e66dc602de6ba9789a0b828a2ad7e5b0047631 authored over 5 years ago

Use the materialized view ooexpl_wc_input_counts

This is related to: https://github.com/ooni/pipeline/pull/222

296158392a4042459e9d0af4c9d0ad5411df850c authored over 5 years ago

Add materialized table to improve OONI Explorer performance

8fb0dd9b3b63d52a89917de81864d61451689b28 authored over 5 years ago

Merge pull request #220 from ooni/tox

Use Tox from Travis

512ca5126046bebf3ce22123679d8298b53d58b9 authored over 5 years ago

Enable basic unit tests and coverage on Travis

69f90c08772e8aa6bd306dead6d5bed3713a88c6 authored over 5 years ago

Extract Postgres port

105b8c89e2576155b0ecdb700627b82f742f9190 authored over 5 years ago

Use Tox from Travis

0fca8db16e1547076cafe551824c1c66a001f11e authored over 5 years ago

Merge pull request #215 from ooni/check-confirmed

Add assert to check if confirmed is > 0

51c98f79019095526f33c367170932631445d5b0 authored over 5 years ago

Add assertion on confirmed count

293acaaa1ce2cfca698f57ce4c41056287536250 authored over 5 years ago

Merge pull request #208 from ooni/metrics

Add pipeline metrics

d4874cf1384eaf70dad38290c43f740d86ebfdfe authored over 5 years ago

Add more metrics

042322a10387e40f0746c12272e5bc8484213299 authored over 5 years ago

Add metrics support

83d1f4735f875c65e56828b9f0338c62aaf98065 authored over 5 years ago

Add comment

85d96e398bc08ed77b027e135ad75dcfa743702c authored over 5 years ago

Add docstrigs

fb8f265f4caeba6debf38c67bd85c103cdcc22de authored over 5 years ago

Add comment

bb4021f271813eb9ed734d0ef28129f8a144b6bd authored over 5 years ago

Add some extra comments suggested by @bassosimone

fb84d3105b52ab3aedf920a4f9539dfffb006af2 authored over 5 years ago

Improve test_integration

Add tox.ini

31bcfbcf2af1a293b5bf2421e94811d7d1f942eb authored over 5 years ago

Bump version add changelog

311239460b4a18774c94371a670ebf177369095a authored over 5 years ago

Merge pull request #102 from ooni/fix-website-urls

Only consider the web_connectivity measurements when calculating webs…

1db5a00f9ed011bb5e779d001a81ee37697a080a authored over 5 years ago

Merge pull request #100 from ooni/feature/domain-search

Add support for searching by domain

1f7789bf427f5e39fd5bde861aac3940902b03fe authored over 5 years ago

Only consider the web_connectivity measurements when calculating website stats

3a31206afc234d7f1302dd722b86803ff6a4e179 authored over 5 years ago

Merge pull request #101 from ooni/better-exceptions

Better exceptions

eec4a0ddf08aab9bb781cb9263f3c473356942c7 authored over 5 years ago

Throw BadRequest instead of Exception to avoid polluting sentry

25bd5ef9d053899c49549ca52b1bff7fa1573e44 authored over 5 years ago

Merge pull request #216 from ooni/fixture-based-testing

Fixture based testing

18165f06468fc3d8b905372b3b74651d96e3fed2 authored over 5 years ago

Add support for searching by domain

This implements: https://github.com/ooni/api/issues/52

7efb334a68c610aaf75d215dedeeac5621e426ad authored over 5 years ago

Add comments about the sections of the private endpoints

dca660539979b776591ef3dcca9cdcf7f9a44cd6 authored over 5 years ago

Use repr instead of str

4bb42e317e7b86da0a27822647dafeed3b0d0541 authored over 5 years ago

Add comment and drop print debug line

3091336101ad51985838cfb3107187500710667e authored over 5 years ago

Fix bug in marking of confirmed and anomaly flags

The measurement_meta and report_blob tables are only filled with data on
ingestion and re-ingest...

24d4424e1f63ac45337baff309da7b817181102d authored over 5 years ago

Merge pull request #219 from ooni/shovel/readme

Add some notes about the shovel

d0be977b7db30de83f7bbfcd9640348b86fbb6cf authored over 5 years ago

Print the http_request_fp table count

f301e1d4c5a88d3a9386bf6969b9b8fc2ec5a909 authored over 5 years ago

Fix bug in centrifugation

b1d55580bd7cab08f2af313ef3f006d9f2b5c956 authored over 5 years ago

Handle header values which are base64 encoded

75db59c375c85027b7188625cbcae04f87f40e4f authored over 5 years ago

Add some notes about the shovel

de34c445905135c34adc680fef709bc103c5ead2 authored over 5 years ago

Add some more debug output

fd8d20e82ae937f0cdddad31933cc5aab4f954ab authored over 5 years ago

Add a fast path for the fingerprints

6d0e92963519b754f1b1bcf21f1f59f10bf97ed6 authored over 5 years ago

Utility functions

* Add function to print query output
* Create working_dir inside of the repo root

d368bc9377beb64fc0e179300e1645e550abf173 authored over 5 years ago

Only set confirmed = NULL

b6c4d5a836c194f91426489f422bcb285bef5db8 authored over 5 years ago

Fix typo in centrifugation

f8e357a4878143f84fe846db0979d5035a2f95e1 authored over 5 years ago

Move tests into Makefile

ecb42518f01b83cecaa6b82ef3f533cf20624dbe authored over 5 years ago

Refactor the integration tests to use a temporary directory

ef9799658c2b3c9f392895c633c3dea0666746e2 authored over 5 years ago

Drop caching as travis fetches from s3 anyways

See: https://docs.travis-ci.com/user/caching/#how-does-caching-work

9ff24d2b1edf169fa444d1cd1add59f08c04b7d4 authored over 5 years ago

Remove the --progress-bar off option from pip

ad1f36056ea41d5a55d8da322fc32e96d48de010 authored over 5 years ago

Merge pull request #218 from ooni/fixture-based-testing-pytest

Fixture based testing pytest

b907e25f7039257b9a887aa617e71422e4b7634a authored over 5 years ago

Disable pip progress bar

ca579770d0f36ed383dc8ef2d3ec8543e145d66e authored over 5 years ago

Switch to Pytest

2f698c41b8810a1301a1f72504e0339bc3099988 authored over 5 years ago

Reformat with Black

8b60f18d16a106f71adaeb56ea370d2c192e8879 authored over 5 years ago

Fix typo

8f1c728b718afb25ce91b900c5418aefb365e185 authored over 5 years ago

Unset the confirmed

d556ad658411b1e9a7f42831c697f19ed89c21db authored over 5 years ago

don't cache

c0b4ff166172ea71026635f9ea7dd8a264e375a8 authored over 5 years ago

Change file

e0de51a5c41ae6c42be6e58fca25c0e09ba63b30 authored over 5 years ago

Create bridge_db.json file

70edecdd909f7210f5d8b0b6168380d71a10c6bc authored over 5 years ago

Fix bugs in the complete workflow

57659619591438cd6f78679f8eec938a9f857eb1 authored over 5 years ago

Only download files if they don't exist

80e5ca2dc721fec370bfbd55cae80ab99795b534 authored over 5 years ago

Fix missing docker_client

32458288017911f655c386139c7f5d45af94881a authored over 5 years ago

More debug output

b8330d548d656ead40976a2e6a4c512dce977a65 authored over 5 years ago

WIP on full integration tests

665342d50920c266822d514d387db4611271b458 authored over 5 years ago

Drop the 1 day lag from the API queries for Explorer

Even if the pipeline ticks at 24h it's still bad to always put a lag of
24h because if the pipel...

36840fa8e365431d5268e64ba256bcb124df0220 authored over 5 years ago

Bump version add changelog entry

7ba9d53cd9bf12b3bf8b42d6a59116db893e7c59 authored over 5 years ago

Merge pull request #96 from ooni/legacy-explr-fix

Legacy explr fix

19f4929eaf1deedf1800a6b6be3d25db0c46943a authored over 5 years ago

Apply a limit of 100

1d8967fa2aa3ac4461643d247a4c0369e7622745 authored over 5 years ago

Restrict legacy explorer related queries to a 2 month window

47461e91eef18f0e2594bb469e99ecd7a2b2056f authored over 5 years ago

Merge pull request #207 from ooni/fingerprint-by-cc

Match fingerprints by country and globally

342da8921fec4bdc2a7f4d09d75d86fb87598f14 authored over 5 years ago

Match fingerprints by country and globally

5e491d720abe9d6f3f649926738d69a5ab62dc94 authored over 5 years ago

Run Black --fast on Python files

0b6ff05f275dcad305c5a98bb3a3755d34860562 authored over 5 years ago

/api/v1/{bouncer,test-helpers}: no duplicates (#123)

Closes #120

Lists are usually small (~3-5 elements), therefore I didn't felt
entitled to do ...

1164a700b63602e82939a2f7e7710277c4e7fc85 authored over 5 years ago

README.rst: fix travis URL after rename (#121)

248548de5fbc07f96484aa8afa9cf08fff4fbad0 authored over 5 years ago

Bump version

592859b4ae70f313198763d96aa40112cd0f1b2c authored over 5 years ago

Merge pull request #94 from ooni/fix-blockpage

Fix blockpage

02f22d0a973d13b005e9e92988d103619c71d0b1 authored over 5 years ago

Apply ordering on test_day and GROUP BY

c66ce09adcda760f67976f590b4f65978e13eedf authored over 5 years ago

Create man dirs

9b1be55035e595dad7308d76a8d3d87dfb3ff2b1 authored over 5 years ago

Use 3.5-slim for python

9107dc4691c864383245aebb7747bc20929d6bf9 authored over 5 years ago

Bump changelog date

b6a048d7adcfe5982c70480777ff1e2ce06f9066 authored over 5 years ago

Bump API version and add Changelog

0d71dc64345f00c884b3980637146df87b7dcf34 authored over 5 years ago

Merge pull request #92 from ooni/performance

Compute the blockpage counts from the mat view

0438e3a8a1e856f90dadc90ad2dcf98ce31c273d authored over 5 years ago

Give better column names

187644683ef050d1369196d33a305eaa535481b6 authored over 5 years ago

Fix ooexpl_wc_confirmed query

Add JOIN on autoclaved, do GROUP BY on test_day, use colum names in
GROUP BY

0a2af2d47877fb18dd81120d585aea494d09b9ac authored over 5 years ago

Add task for refreshing the ooni-explorer web_connectivity mat view

e73ddd340b655fc079794991deca6e68ecd1695e authored over 5 years ago

Move from ooexpl_wc_confirmed view to table

Insert one day at a time to improve efficiency.
In case of duplicate runs of centrifugation with...

4c4a121f64539be736c98cc1610ffa3297ae2d58 authored over 5 years ago

Add one-shot INSERT to comment of migration script

08a17f269fc515800ef968dccb7a8082d5430bbe authored over 5 years ago

Drop query from comment

0b39ccc283df711091b2354daa1cd62c9019353b authored over 5 years ago

Add migration scripts for the mat view

350a24de75d406215b45b111abb0c60352608ee6 authored over 5 years ago

INSERT filtering on bucket_date

7b7b166d44f659333b65fddb82c73016e34d16e9 authored over 5 years ago

Compute the blockpage counts from the mat view

xref: https://github.com/ooni/pipeline/pull/198

8a3300698b7718a07758becb2a9498045d86f1ef authored over 5 years ago

Merge pull request #88 from ooni/performance

Performance

81cf7b7f7cbbcf046497c35bb76350d8c9bbd283 authored almost 6 years ago

Merge pull request #89 from pellaeon/master

Readme: add building assets

7ba9160eb48264e9f120724a9e064469230a0b94 authored almost 6 years ago

Readme: add building assets

9a70f54b452b9970317f64103430d368c9eb076f authored almost 6 years ago

Merge pull request #183 from ooni/doc-reprocessing

Doc on reprocessing & reingestion

274eeecbb6ac64c99cdd89e0407e69314cd81e85 authored almost 6 years ago

Merge branch 'master' into doc-reprocessing

* master: (67 commits)
Fix typo in function name
Run check for reingestion as well
Check f...

3d0ea58e8ab9a6ff7e0ff2160ec076f71a85702a authored almost 6 years ago

Add example of editing PRJ

5208ff2c144cf09e66e510ea279e55f44702e9fc authored almost 6 years ago

Add example on code_ver bump

2d075dbe31b6a09c1c6e3beca3297ce056247df3 authored almost 6 years ago