github.com/ooni/backend commits | Ecosyste.ms: OpenCollective

Read collector hostnames from conffile

211ca8f9a338e5e75c296a2355edcf98d736c595 authored over 5 years ago

Update table permissions

2cfd6319e2d8cb67ca51a01913ed8a5deb38f1a4 authored over 5 years ago

Add comment on slow query

77948d9498348b50fedc865879fd8269bac62d96 authored over 5 years ago

Merge faspath and traditional measurements

3e053de7232e2bf7ed890e033bedd8d0f8de8127 authored over 5 years ago

Improve query logging

5dc9f9b187471e8c178dd4225b7ed8f6a208ee63 authored over 5 years ago

More fine grained tests on fastpath vs traditional

afaa31374dc44f3e04807a2fd38e939388d5b3b8 authored over 5 years ago

Improve logging

8a2eec1513125a0658707e6ec4a391203a7bd000 authored over 5 years ago

Bump urllib3 from 1.24.1 to 1.24.2 in /tests

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.24.1 to 1.24.2.
- [Release notes](htt...

bc7b86231204b08e8d78feea6e7d75a38290dfdb authored over 5 years ago

Fix file creation permissions

a1470d7836a1101c7ce600285d6eb7eb19de0fb9 authored over 5 years ago

Switch to simpler queue implementation

5027244d9378f49d45d599033adb438734b473e0 authored over 5 years ago

Cleanup

ca2e1f41419efdacbfd950f50ec22b5c9cf196ce authored over 5 years ago

Normalize: add tests, disable on fastpath

ea660edc353e01801449350d4eef4e41a3a45c34 authored over 5 years ago

Update fastpath measurement fetching

c80e158f647711d649abdc2b4ea4501c9da70e29 authored over 5 years ago

Improve pytest/Flask logging in integ tests

074ecfe86385f3a164dae9e3571b2a19de77f6c2 authored over 5 years ago

Update FASTPATH_MSM_ID_PREFIX

12f57d3696e8e729fdc79b9264e56d2f936b60e5 authored over 5 years ago

Drop fingerprints CSV

63103dac63c4898f99320b0c12da2d6118cb858e authored over 5 years ago

Cleanup

fab45fff1b259d0bc1a376d782d2347c806f72d3 authored over 5 years ago

Run only on web_connectivity

b7ed9f696876e306f0d0bc378db3cb73de31bbf1 authored over 5 years ago

Cleanup normalize, improve tests

be6294da79cadcfb246a09747b9dc5468f9f2712 authored over 5 years ago

Handle ZZ fingerprints, add tests

6f3634948595f231f07740b9d7310b792f8efffe authored over 5 years ago

Update dependencies, minor cleanup

ec66d34fdd84b520481dbb30c8dd88db312b9e31 authored over 5 years ago

Add support for writing into the platform column in the fastpath table

cfad9ff11c3a54018d3a74ded801ab242580840b authored over 5 years ago

Add a portable Queue implementation to work on macOS

See: https://github.com/vterron/lemon/commit/9ca6b4b

125eeeb701f86ec7cb56bd19e01231a5a12a3d48 authored over 5 years ago

Make the import of systemd conditional to make it work on macOS

ba39ca09da1f78e10ee3de91eebcf2ea0ce15344 authored over 5 years ago

Add requirements.txt file

36278fe617416835f7753bf5dab4f95e9e40a2aa authored over 5 years ago

Leave more free space

d6374f5ce0401c26a7ae4f812b6334a39ee87074 authored over 5 years ago

Move unused code out

755bcb7c34f5b99b6b879de69eaefe4868726d43 authored over 5 years ago

Send NOTIFY, increase trimming, minor cleanup

06dac0da5ce3d06a80e85d6266257da2a83fcb0f authored over 5 years ago

Cleanup unused imports

916e2904e67696ff1500219ab96f11aac0f63b79 authored over 5 years ago

Give access to scores column

Minor fixes

2ea82c56c2a0be9fbc4fa58f83c470431639b824 authored over 5 years ago

Trim old measurements

3245c78bddc21e5b195e7a394e831b85d0982e23 authored over 5 years ago

Add fastpath support and integration tests

Move the generation of measurement_id into the db query

27ec45d19b5b17bad614f201bbc867d9f4438276 authored over 5 years ago

Add fastpath db model

e8c252ac4ea8771073f96a1325a37cc8c6bd9ea8 authored over 5 years ago

Update country list

f1c3c79b0fb11ce08ae69d2b3d1b6904ad9e8900 authored over 5 years ago

Reflow file

08b6434f6af20165a6e41f11cd9f4be72d6fff40 authored over 5 years ago

Add tox.ini and update makefile

e512b0e137fd1e10d89528f3e2407f1f28ed8b34 authored over 5 years ago

Handle exceptions in sshfeeder

Simplify code

a78c7593b3612715d34883f4470e1643edc3bb10 authored over 5 years ago

Bugfix

724db577074f9c8dcce4f8f7a4d16171ca5ed1f8 authored over 5 years ago

Improve SFTP metrics

ffdd2bb9ba395bbc3b76a4dbb2f6985d7d414cfa authored over 5 years ago

Update README

f2496a078049d6274188c1978b92b1fb7f6665da authored over 5 years ago

Enable DB ping

6bd07aee956bae476645789ae5785397ed38f3f7 authored over 5 years ago

Drop unneded SSH caching

ea1b5a0cdb28861883b3990821b2620d10ece3b4 authored over 5 years ago

Handle exceptions, improve reliability and logging

be99b482a26906778786cf459d05e803528c95db authored over 5 years ago

Update YAML handling

880f6e8f9d215807c23b2d6d8a7ce08080747808 authored over 5 years ago

Add comment

b97fd0a3600e675182c0d9c7cfd900f1fef167e7 authored over 5 years ago

Handle missing archive directory

adbaa1f32a0fce793cc9c16b3d0596a59e55a36c authored over 5 years ago

Add comment

7774d1ed8e278f2e78fc6c658ec546561f49e420 authored over 5 years ago

Add YAML support

Better exception logging
Rename report -> measurement
Handle body_proportion being None

172eca329ee2022d8c069d919a0350e23668c8d6 authored over 5 years ago

Bump versions

a1817780202a6d56a4634bbe520050231fa01a2c authored over 5 years ago

Improvements to the SQL query

8a5cc9ee93b7cf90f577b0e9870284d51fe76f5d authored over 5 years ago

Sort keys and unit test

aa85a7958af9f1ac64300c7f71aef7cc5af9c1dc authored over 5 years ago

Count exceptions

a0dda6ab1a2375800f3a251fa63eacb8f225f872 authored over 5 years ago

Update test

9a97a31ca35d81b34e1da2d411cf84391b2b8759 authored over 5 years ago

Revert "Work on optimizing the list_measurements API endpoints"

This reverts commit 4807a4f0aa4d0efd88bba00b349bbfdc649aab82.

bc21fd735ab2510315a78a4c04bbece3c52919b3 authored over 5 years ago

Add trivial ID, prevent race in writing files

Improve logging

18b2a46ffd23194c7ee7b5074247e01ebd720ef8 authored over 5 years ago

Fix body_proportion bug

7173b2efba3927bdb2a596a67eadaccc141a35bd authored over 5 years ago

Consolidate tables, add flag to overwrite outputs

Cast NULL input into ""
Detect, log and report collisions

1f15622789a46749314a18e559de1550ede3a743 authored over 5 years ago

Remove JSON parsing from sshfeeder as well

bc7dde8b619f97726c64f7241535be29b28fce2f authored over 5 years ago

Bump version

f4fa57e68137e046760bb9235fcd21d6d3ac39d2 authored over 5 years ago

Add Changelog entry

bb1324aa99f4b14f8dc7c5fc9d925c3bd9993481 authored over 5 years ago

Upgrade Werkzeug

86a89460ef4c31449f122951e0b452410dfb491b authored over 5 years ago

Add Postgres integration

Implement multiprocessing wth 15 workers and a queue
Write measurement to disk
s3feeder: move JS...

0f77bf7e91aa35b1ec6a2808896c2954aa5019d5 authored over 5 years ago

Rename module

053346e6260871bab6bc50e22e6a406bc7088486 authored over 5 years ago

Move the by_month queries into a separate call

c100cea4d13fd574b1620613cbd580c0679a39ec authored over 5 years ago

Fix typo in jsonify

46f4461e13fa2d02505c05466598e1a7a7132a6d authored over 5 years ago

Restore private runs_by_month endpoint to how it was

fd47d5006db00627cfaea0c6de65e5e30b526939 authored over 5 years ago

Fix of by one with last bucket

35b44f422be4818579b7c9a00b48ce8508675971 authored over 5 years ago

Optimize the api_private_stats endpoints

e34108384fa1dec1d48f597b69aaf4795051443d authored over 5 years ago

Add the by_month stats to the global_overview endpoint

c515cacebd9f8c643cc618e4d4c5729f46df779c authored over 5 years ago

Merge pull request #107 from ooni/bugfix-testday

Fix the query to avoid dupe dates

b98a117e49e4e85cd12267f0fbf5601ba35f871b authored over 5 years ago

Fix the query to avoid dupe dates

835eb4720fbed913bc7a4511eeafec0199bd4a28 authored over 5 years ago

Merge pull request #105 from ooni/faster-domain-filter

Faster domain filter

cbf7a3925812d01af798e70cb3f71de153dffe2f authored over 5 years ago

Add basic smoke test for input_cte

cfe9bf321385cc193dbb185fbbb3c260acf541b0 authored over 5 years ago

Drop unused imports

ae7f2b1f78f9d924d67535fd738265e1bae27cfe authored over 5 years ago

Minor change

04266d958e392a689bc8faa34875a71ae717a29b authored over 5 years ago

Work on optimizing the list_measurements API endpoints

Query time for an example query has gone from 537,136.721 ms
to 146.392 ms.

This improvement is...

4807a4f0aa4d0efd88bba00b349bbfdc649aab82 authored over 5 years ago

Don't send sentry exceptions while in developmnet mode

3d780929381d15afa0597679cbad53b1f984aa1d authored over 5 years ago

Add more debug to query time logging

abd4a142624373795290407acba4900e84e84b93 authored over 5 years ago

Restructure to allow uploading msmt to S3

bb3a25361a09557f6427cf1cf577ad5229483191 authored over 5 years ago

Rename module

5744ccf9b59c3910bf15e13e732b6f70b8dffe90 authored over 5 years ago

Rename report -> measurement where needed

9936a31bb5abd2073c428437ca4a2688798e71d0 authored over 5 years ago

Cleanup status reset and add test

ac048978aa2c817fd7ac53dd8ada2c2fb1e7fd40 authored over 5 years ago

Bugfix

15d02bd268e1c8277652eaa8f21bd4a27681094e authored over 5 years ago

Estimate run ETA

ab83287ee258d6f7bd229924f3020bee6bb65986 authored over 5 years ago

Bugfix

c369f0c4179aa1989c62cc96a533422e2b5e0071 authored over 5 years ago

Add comments

68bd9177b00f97e88b562a6f0c1548bb5f4c6be3 authored over 5 years ago

Add S3 file uploader

3b67afc452591ca143389eef305a9ddf0ee2b2ef authored over 5 years ago

Bump version add changelog

c008868db22735aa3de076998342f2d3af864cf1 authored over 5 years ago

Merge pull request #103 from ooni/ooexpl_wc_input_counts

Use the materialized view ooexpl_wc_input_counts

8f1648f29f1ef7af9766cebb67679b1caf20b79f authored over 5 years ago

Add missing column to select

6a60d0ed5d4bdd11dc13258876b4e095f1ce6a44 authored over 5 years ago

Fix typo

83aece04bd2ed868fd3833036d658203ca05ff0c authored over 5 years ago

Fix passing Makefile variable

b6bcea7a3dcdd2404c126c0af6592549bd2e5277 authored over 5 years ago

Run the travis tests with production mode on

c2a4a10807f04ccca7889901e6c596ac3e6e5ce9 authored over 5 years ago

Use the pipeline master for fixtures

d7764a2d08e0f16dd6487207b27e95d6c606285d authored over 5 years ago

Add a fastpath for blockpages

2f1aa8c1bc045179f8bed3e7e3732713e2e7e8f4 authored over 5 years ago

Minor cleanup

bbd97090db6aa087adf4ba9019fb1f7d40d6d024 authored over 5 years ago

Sort by total_count before anomaly count

9a340a2045690cd62281c796c280d65ea6703251 authored over 5 years ago

Add failure_count

9d15ac78e5636ea890f245010e214a7f1052a0f6 authored over 5 years ago

Sort cans before loading

b920a425c54402938c4048abe833c618ae5df374 authored over 5 years ago

Switch fingerprinting and away from Pandas

Improve performance
Preprocess fingerprints
Improve metrics

Comment out WIP/unused code

f5314b99ab483602808206386f088213cb8b95ee authored over 5 years ago